2024-08-19T13:10:20.463 INFO:root:teuthology version: 1.2.0 2024-08-19T13:10:20.481 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-19T13:10:20.572 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi/7859586 branch: squid description: rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} tasks/libcephsqlite} email: ceph-qa@ceph.com first_in_suite: false job_id: '7859586' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: debug cephsqlite: 20 debug client: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 3 ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - POOL_APP_NOT_ENABLED - do not have an application enabled sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 78215f5a1cf5eaf321d506a66539942e12851f34 owner: scheduled_teuthology@teuthology priority: 700 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 3241 sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 sleep_before_teardown: 0 subset: 18710/100000 suite: rados suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 78215f5a1cf5eaf321d506a66539942e12851f34 targets: smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIRjIVoA1i5YJhna8z+/nKH6ZE4K6G3Em9WsOE+j6N6m+VaSRIpQ6oLuTmC//A7MaBv6h0IbMSUWknNbWv/KT38= smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNJpH5obQCcY3ccTGPYbIDaDbAgIFhw+7EvrnGx1UzW3YvudX13ku8x9Y2fGckrhgbSMmYQDFybwaM9IeDWFeAA= tasks: - install: extra_system_packages: deb: - sqlite3 rpm: - sqlite-devel - ceph: null - exec: client.0: - ceph osd pool create cephsqlite - ceph auth get-or-create client.libcephsqlite mon 'profile simple-rados-client-with-blocklist' osd 'allow rwx pool=cephsqlite' >> /etc/ceph/ceph.keyring - exec: client.0: - ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr - workunit: clients: client.0: - rados/test_libcephsqlite.sh cephsqlite env: CEPH_ARGS: --id libcephsqlite --no-log-to-stderr teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 67db276914d29505c9707d04f3a3b8fb9d8aee4b timestamp: 2024-08-18_21:00:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-19T13:10:20.573 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa; will attempt to use it 2024-08-19T13:10:20.573 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks 2024-08-19T13:10:20.573 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-19T13:10:20.575 INFO:teuthology.task.internal:Checking packages... 2024-08-19T13:10:20.597 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'de09dd018de4baddfa6ca4d43bc8fb784cf75833' 2024-08-19T13:10:20.597 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-19T13:10:20.598 INFO:teuthology.packaging:ref: None 2024-08-19T13:10:20.598 INFO:teuthology.packaging:tag: None 2024-08-19T13:10:20.598 INFO:teuthology.packaging:branch: squid 2024-08-19T13:10:20.598 INFO:teuthology.packaging:sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T13:10:20.598 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-08-19T13:10:20.801 INFO:teuthology.task.internal:Found packages for ceph version 19.1.0-1386.g78215f5a 2024-08-19T13:10:20.803 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-19T13:10:20.810 INFO:teuthology.task.internal:no buildpackages task found 2024-08-19T13:10:20.810 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-19T13:10:20.908 INFO:teuthology.task.internal:Saving configuration 2024-08-19T13:10:20.921 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-19T13:10:20.962 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-19T13:10:20.993 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi/7859586', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-19 13:04:52.843518', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIRjIVoA1i5YJhna8z+/nKH6ZE4K6G3Em9WsOE+j6N6m+VaSRIpQ6oLuTmC//A7MaBv6h0IbMSUWknNbWv/KT38='} 2024-08-19T13:10:21.026 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi/7859586', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-19 13:04:52.845197', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNJpH5obQCcY3ccTGPYbIDaDbAgIFhw+7EvrnGx1UzW3YvudX13ku8x9Y2fGckrhgbSMmYQDFybwaM9IeDWFeAA='} 2024-08-19T13:10:21.026 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-19T13:10:21.033 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-08-19T13:10:21.034 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-08-19T13:10:21.034 INFO:teuthology.run_tasks:Running task console_log... 2024-08-19T13:10:21.187 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcb2f557ac0>, signals=[15]) 2024-08-19T13:10:21.187 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-19T13:10:21.193 INFO:teuthology.task.internal:Opening connections... 2024-08-19T13:10:21.194 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T13:10:21.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T13:10:21.278 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-08-19T13:10:21.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T13:10:21.362 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-19T13:10:21.369 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-08-19T13:10:21.387 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-08-19T13:10:21.387 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-08-19T13:10:21.447 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-08-19T13:10:21.447 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-08-19T13:10:21.447 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-08-19T13:10:21.447 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-08-19T13:10:21.447 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-08-19T13:10:21.447 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-08-19T13:10:21.447 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-19T13:10:21.447 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-08-19T13:10:21.447 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-08-19T13:10:21.447 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-19T13:10:21.447 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-08-19T13:10:21.448 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-19T13:10:21.448 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-19T13:10:21.448 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-19T13:10:21.449 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-08-19T13:10:21.473 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-08-19T13:10:21.491 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-08-19T13:10:21.492 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-08-19T13:10:21.549 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-08-19T13:10:21.550 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-08-19T13:10:21.550 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-08-19T13:10:21.550 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-08-19T13:10:21.550 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-08-19T13:10:21.550 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-08-19T13:10:21.550 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-19T13:10:21.550 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-08-19T13:10:21.550 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-08-19T13:10:21.550 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-19T13:10:21.550 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-08-19T13:10:21.550 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-19T13:10:21.550 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-19T13:10:21.550 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-19T13:10:21.551 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-08-19T13:10:21.581 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-19T13:10:21.591 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-19T13:10:21.611 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-19T13:10:21.612 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-08-19T13:10:21.614 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-08-19T13:10:21.630 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-19T13:10:21.636 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-19T13:10:21.637 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-08-19T13:10:21.672 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-08-19T13:10:21.707 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-19T13:10:21.777 INFO:teuthology.run_tasks:Running task kernel... 2024-08-19T13:10:21.791 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-08-19T13:10:21.791 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-08-19T13:10:21.792 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-19T13:10:21.792 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-19T13:10:21.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T13:10:21.809 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-08-19T13:10:21.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T13:10:21.810 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-08-19T13:10:21.867 INFO:teuthology.orchestra.run.smithi122.stdout:5.14.0-480.el9.x86_64 2024-08-19T13:10:21.867 INFO:teuthology.task.kernel:Running kernel on smithi122: 5.14.0-480.el9.x86_64 2024-08-19T13:10:21.868 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum install -y kernel 2024-08-19T13:10:21.869 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-480.el9.x86_64 2024-08-19T13:10:21.869 INFO:teuthology.task.kernel:Running kernel on smithi103: 5.14.0-480.el9.x86_64 2024-08-19T13:10:21.869 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum install -y kernel 2024-08-19T13:10:24.233 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-08-19T13:10:24.238 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-08-19T13:10:26.048 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-08-19T13:10:28.875 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 20 MB 00:03 2024-08-19T13:10:30.807 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.6 MB 00:00 2024-08-19T13:10:32.216 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 104 kB/s | 18 kB 00:00 2024-08-19T13:10:33.212 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 26 MB/s | 22 MB 00:00 2024-08-19T13:10:35.276 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.6 MB 00:02 2024-08-19T13:10:37.293 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 18 kB 00:00 2024-08-19T13:10:38.322 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 25 MB/s | 22 MB 00:00 2024-08-19T13:10:38.701 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-08-19T13:10:40.640 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-08-19T13:10:40.641 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-480.el9.x86_64 is already installed. 2024-08-19T13:10:40.719 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-08-19T13:10:40.720 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-08-19T13:10:40.720 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-08-19T13:10:40.826 DEBUG:teuthology.orchestra.run.smithi122:> echo no | sudo yum reinstall kernel || true 2024-08-19T13:10:41.348 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:03 ago on Mon 19 Aug 2024 01:10:38 PM UTC. 2024-08-19T13:10:41.518 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-08-19T13:10:41.524 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-08-19T13:10:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: Package Architecture Version Repository Size 2024-08-19T13:10:41.524 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-08-19T13:10:41.524 INFO:teuthology.orchestra.run.smithi122.stdout:Reinstalling: 2024-08-19T13:10:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-08-19T13:10:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:10:41.525 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-08-19T13:10:41.525 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-08-19T13:10:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:10:41.525 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 1.5 M 2024-08-19T13:10:41.525 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 0 2024-08-19T13:10:41.526 INFO:teuthology.orchestra.run.smithi122.stderr:Operation aborted. 2024-08-19T13:10:41.585 INFO:teuthology.orchestra.run.smithi122.stdout:Is this ok [y/N]: 2024-08-19T13:10:41.585 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum reinstall -y kernel || true 2024-08-19T13:10:42.102 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:04 ago on Mon 19 Aug 2024 01:10:38 PM UTC. 2024-08-19T13:10:42.223 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-08-19T13:10:42.224 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-08-19T13:10:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: Package Architecture Version Repository Size 2024-08-19T13:10:42.224 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-08-19T13:10:42.224 INFO:teuthology.orchestra.run.smithi122.stdout:Reinstalling: 2024-08-19T13:10:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-08-19T13:10:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:10:42.225 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-08-19T13:10:42.225 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-08-19T13:10:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:10:42.225 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 1.5 M 2024-08-19T13:10:42.225 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 0 2024-08-19T13:10:42.225 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-08-19T13:10:43.828 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-08-19T13:10:44.203 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-480.el9.x86_64.rpm 825 kB/s | 1.5 MB 00:01 2024-08-19T13:10:44.203 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-08-19T13:10:44.203 INFO:teuthology.orchestra.run.smithi122.stdout:Total 783 kB/s | 1.5 MB 00:01 2024-08-19T13:10:44.235 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-08-19T13:10:44.483 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-08-19T13:10:44.483 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-08-19T13:10:44.508 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-08-19T13:10:44.508 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-08-19T13:10:44.666 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-08-19T13:10:44.771 INFO:teuthology.orchestra.run.smithi122.stdout: Reinstalling : kernel-5.14.0-480.el9.x86_64 1/2 2024-08-19T13:10:44.771 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T13:10:45.559 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T13:10:45.560 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 1/2 2024-08-19T13:10:45.734 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-08-19T13:10:45.734 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-480.el9.x86_64 is already installed. 2024-08-19T13:10:45.808 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-08-19T13:10:45.809 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-08-19T13:10:45.810 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-08-19T13:10:45.903 DEBUG:teuthology.orchestra.run.smithi103:> echo no | sudo yum reinstall kernel || true 2024-08-19T13:10:46.298 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T13:10:46.298 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:10:46.298 INFO:teuthology.orchestra.run.smithi122.stdout:Reinstalled: 2024-08-19T13:10:46.298 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-5.14.0-480.el9.x86_64 2024-08-19T13:10:46.298 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:10:46.298 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-08-19T13:10:46.425 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:03 ago on Mon 19 Aug 2024 01:10:43 PM UTC. 2024-08-19T13:10:46.501 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel | sort -rV | head -n 1 2024-08-19T13:10:46.557 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-480.el9.x86_64 2024-08-19T13:10:46.558 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-08-19T13:10:46.558 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-08-19T13:10:46.558 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-08-19T13:10:46.558 DEBUG:teuthology.orchestra.run.smithi122:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-19T13:10:46.572 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-08-19T13:10:46.585 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-08-19T13:10:46.585 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-08-19T13:10:46.585 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-08-19T13:10:46.585 INFO:teuthology.orchestra.run.smithi103.stdout:Reinstalling: 2024-08-19T13:10:46.586 INFO:teuthology.orchestra.run.smithi103.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-08-19T13:10:46.586 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:10:46.586 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-08-19T13:10:46.586 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-08-19T13:10:46.586 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:10:46.586 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 1.5 M 2024-08-19T13:10:46.586 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 0 2024-08-19T13:10:46.586 INFO:teuthology.orchestra.run.smithi103.stderr:Operation aborted. 2024-08-19T13:10:46.598 INFO:teuthology.orchestra.run.smithi122.stdout:ttyS1 2024-08-19T13:10:46.617 DEBUG:teuthology.parallel:result is None 2024-08-19T13:10:46.641 INFO:teuthology.orchestra.run.smithi103.stdout:Is this ok [y/N]: 2024-08-19T13:10:46.642 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum reinstall -y kernel || true 2024-08-19T13:10:47.158 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:04 ago on Mon 19 Aug 2024 01:10:43 PM UTC. 2024-08-19T13:10:47.280 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-08-19T13:10:47.280 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-08-19T13:10:47.280 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-08-19T13:10:47.281 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-08-19T13:10:47.281 INFO:teuthology.orchestra.run.smithi103.stdout:Reinstalling: 2024-08-19T13:10:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-08-19T13:10:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:10:47.281 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-08-19T13:10:47.281 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-08-19T13:10:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:10:47.281 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 1.5 M 2024-08-19T13:10:47.281 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 0 2024-08-19T13:10:47.281 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-08-19T13:10:47.606 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-480.el9.x86_64.rpm 6.7 MB/s | 1.5 MB 00:00 2024-08-19T13:10:47.606 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-08-19T13:10:47.606 INFO:teuthology.orchestra.run.smithi103.stdout:Total 4.7 MB/s | 1.5 MB 00:00 2024-08-19T13:10:47.642 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-08-19T13:10:47.882 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-08-19T13:10:47.882 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-08-19T13:10:47.907 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-08-19T13:10:47.907 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-08-19T13:10:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-08-19T13:10:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: Reinstalling : kernel-5.14.0-480.el9.x86_64 1/2 2024-08-19T13:10:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T13:10:48.888 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T13:10:48.889 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 1/2 2024-08-19T13:10:49.656 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T13:10:49.656 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:10:49.656 INFO:teuthology.orchestra.run.smithi103.stdout:Reinstalled: 2024-08-19T13:10:49.656 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-5.14.0-480.el9.x86_64 2024-08-19T13:10:49.656 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:10:49.656 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-08-19T13:10:49.843 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q kernel | sort -rV | head -n 1 2024-08-19T13:10:49.900 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-480.el9.x86_64 2024-08-19T13:10:49.900 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-08-19T13:10:49.900 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-08-19T13:10:49.901 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-08-19T13:10:49.901 DEBUG:teuthology.orchestra.run.smithi103:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-19T13:10:49.944 INFO:teuthology.orchestra.run.smithi103.stdout:ttyS1 2024-08-19T13:10:49.964 DEBUG:teuthology.parallel:result is None 2024-08-19T13:10:49.964 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-19T13:10:49.973 INFO:teuthology.task.internal:Creating test directory... 2024-08-19T13:10:49.973 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-19T13:10:50.007 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-19T13:10:50.026 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-19T13:10:50.032 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-19T13:10:50.123 INFO:teuthology.task.internal:Creating archive directory... 2024-08-19T13:10:50.123 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-19T13:10:50.125 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-19T13:10:50.182 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-19T13:10:50.190 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-19T13:10:50.190 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-19T13:10:50.192 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-19T13:10:50.220 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T13:10:50.228 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T13:10:50.230 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T13:10:50.240 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T13:10:50.242 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-19T13:10:50.249 INFO:teuthology.task.internal:Configuring sudo... 2024-08-19T13:10:50.250 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-19T13:10:50.275 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-19T13:10:50.313 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-19T13:10:50.323 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-19T13:10:50.323 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-19T13:10:50.340 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-19T13:10:50.372 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T13:10:50.444 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T13:10:50.487 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T13:10:50.575 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T13:10:50.609 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:10:50.609 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-19T13:10:50.674 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T13:10:50.728 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T13:10:50.783 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T13:10:50.837 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T13:10:50.873 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:10:50.874 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-19T13:10:50.944 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-08-19T13:10:50.946 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-08-19T13:10:50.976 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T13:10:51.028 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T13:10:51.502 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-19T13:10:51.510 INFO:teuthology.task.internal:Starting timer... 2024-08-19T13:10:51.510 INFO:teuthology.run_tasks:Running task pcp... 2024-08-19T13:10:51.521 INFO:teuthology.run_tasks:Running task selinux... 2024-08-19T13:10:51.530 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-08-19T13:10:51.530 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-08-19T13:10:51.610 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-08-19T13:10:51.612 DEBUG:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2024-08-19T13:10:51.681 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: 2024-08-19T13:10:51.683 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-19T13:10:51.683 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-08-19T13:10:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-08-19T13:10:51.704 DEBUG:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2024-08-19T13:10:51.757 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2024-08-19T13:10:51.757 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi103.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi122.front.sepia.ceph.com': 'permissive'} 2024-08-19T13:10:51.757 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T13:10:51.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T13:10:51.783 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T13:10:51.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T13:10:51.812 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-19T13:10:51.812 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-08-19T13:10:51.847 DEBUG:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2024-08-19T13:10:51.879 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-19T13:10:51.888 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-19T13:10:51.901 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-19T13:10:51.902 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi103.front.sepia.ceph.com,smithi122.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-19T13:12:36.572 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi103.front.sepia.ceph.com'), Remote(name='ubuntu@smithi122.front.sepia.ceph.com')] 2024-08-19T13:12:36.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-08-19T13:12:36.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T13:12:36.663 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-08-19T13:12:36.746 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-08-19T13:12:36.750 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-08-19T13:12:36.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T13:12:36.834 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-08-19T13:12:36.918 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-08-19T13:12:36.918 INFO:teuthology.run_tasks:Running task clock... 2024-08-19T13:12:36.929 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-19T13:12:36.929 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-19T13:12:36.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T13:12:36.932 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-19T13:12:36.933 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T13:12:36.959 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-19T13:12:36.973 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-19T13:12:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-19T13:12:36.998 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-08-19T13:12:37.003 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-19T13:12:37.008 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-08-19T13:12:37.023 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-19T13:12:37.028 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2024-08-19T13:12:37.037 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-19T13:12:37.039 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2024-08-19T13:12:37.053 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-19T13:12:37.067 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-19T13:12:37.095 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-08-19T13:12:37.100 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T13:12:37.100 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-08-19T13:12:37.100 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T13:12:37.100 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T13:12:37.100 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T13:12:37.101 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T13:12:37.122 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-08-19T13:12:37.125 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T13:12:37.125 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-08-19T13:12:37.125 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T13:12:37.126 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T13:12:37.126 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T13:12:37.126 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T13:12:37.127 INFO:teuthology.run_tasks:Running task install... 2024-08-19T13:12:37.134 DEBUG:teuthology.task.install:project ceph 2024-08-19T13:12:37.134 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833'}} 2024-08-19T13:12:37.135 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833'} 2024-08-19T13:12:37.135 INFO:teuthology.task.install:Using flavor: default 2024-08-19T13:12:37.141 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-19T13:12:37.142 INFO:teuthology.task.install:extra packages: [] 2024-08-19T13:12:37.142 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'tag': None, 'wait_for_package': False} 2024-08-19T13:12:37.143 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T13:12:37.152 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'tag': None, 'wait_for_package': False} 2024-08-19T13:12:37.153 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T13:12:37.303 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/ 2024-08-19T13:12:37.303 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1382.gde09dd01 2024-08-19T13:12:37.315 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/ 2024-08-19T13:12:37.315 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1382.gde09dd01 2024-08-19T13:12:37.426 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-19T13:12:37.426 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:12:37.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-19T13:12:37.431 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-19T13:12:37.431 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:12:37.431 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-19T13:12:37.454 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, sqlite-devel, sqlite-devel, sqlite-devel on remote rpm x86_64 2024-08-19T13:12:37.454 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/de09dd018de4baddfa6ca4d43bc8fb784cf75833/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-19T13:12:37.457 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, sqlite-devel, sqlite-devel, sqlite-devel, sqlite-devel on remote rpm x86_64 2024-08-19T13:12:37.457 DEBUG:teuthology.orchestra.run.smithi122:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/de09dd018de4baddfa6ca4d43bc8fb784cf75833/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-19T13:12:37.522 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-19T13:12:37.523 DEBUG:teuthology.orchestra.run.smithi122:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-19T13:12:37.598 DEBUG:teuthology.orchestra.run.smithi122:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-19T13:12:37.599 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-19T13:12:37.662 INFO:teuthology.orchestra.run.smithi122.stdout:check_obsoletes = 1 2024-08-19T13:12:37.663 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum clean all 2024-08-19T13:12:37.664 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-08-19T13:12:37.665 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-08-19T13:12:37.890 INFO:teuthology.orchestra.run.smithi122.stdout:45 files removed 2024-08-19T13:12:37.893 INFO:teuthology.orchestra.run.smithi103.stdout:45 files removed 2024-08-19T13:12:37.917 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd sqlite-devel sqlite-devel sqlite-devel sqlite-devel 2024-08-19T13:12:37.923 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd sqlite-devel sqlite-devel sqlite-devel sqlite-devel 2024-08-19T13:12:38.460 INFO:teuthology.orchestra.run.smithi122.stdout:ceph packages for x86_64 310 kB/s | 84 kB 00:00 2024-08-19T13:12:38.478 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 299 kB/s | 84 kB 00:00 2024-08-19T13:12:38.743 INFO:teuthology.orchestra.run.smithi122.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-08-19T13:12:38.763 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-08-19T13:12:39.036 INFO:teuthology.orchestra.run.smithi122.stdout:ceph source packages 7.4 kB/s | 1.9 kB 00:00 2024-08-19T13:12:39.038 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-08-19T13:12:40.169 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 7.5 MB/s | 8.2 MB 00:01 2024-08-19T13:12:40.296 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.2 MB 00:01 2024-08-19T13:12:42.056 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-08-19T13:12:42.487 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-08-19T13:12:46.855 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.6 MB 00:00 2024-08-19T13:12:47.524 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.6 MB 00:01 2024-08-19T13:12:48.506 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 18 kB 00:00 2024-08-19T13:12:49.024 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 18 kB 00:00 2024-08-19T13:12:49.884 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 18 MB/s | 22 MB 00:01 2024-08-19T13:12:50.131 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 23 MB/s | 22 MB 00:00 2024-08-19T13:12:55.441 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-08-19T13:12:55.520 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-08-19T13:12:57.661 INFO:teuthology.orchestra.run.smithi122.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T13:12:57.662 INFO:teuthology.orchestra.run.smithi122.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T13:12:57.722 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T13:12:57.722 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T13:12:57.723 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-08-19T13:12:57.731 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-08-19T13:12:57.731 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-08-19T13:12:57.731 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-08-19T13:12:57.732 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-08-19T13:12:57.732 INFO:teuthology.orchestra.run.smithi122.stdout: ceph x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 6.6 k 2024-08-19T13:12:57.732 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T13:12:57.732 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 896 k 2024-08-19T13:12:57.732 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 145 k 2024-08-19T13:12:57.732 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 1.5 M 2024-08-19T13:12:57.732 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 146 k 2024-08-19T13:12:57.732 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 3.6 M 2024-08-19T13:12:57.732 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 7.4 M 2024-08-19T13:12:57.732 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 49 k 2024-08-19T13:12:57.732 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 11 M 2024-08-19T13:12:57.732 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 49 M 2024-08-19T13:12:57.733 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 285 k 2024-08-19T13:12:57.733 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 766 k 2024-08-19T13:12:57.733 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 32 k 2024-08-19T13:12:57.733 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 753 k 2024-08-19T13:12:57.733 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 127 k 2024-08-19T13:12:57.733 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 188 k 2024-08-19T13:12:57.733 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 369 k 2024-08-19T13:12:57.733 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 350 k 2024-08-19T13:12:57.733 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 117 k 2024-08-19T13:12:57.733 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 85 k 2024-08-19T13:12:57.733 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.1 M 2024-08-19T13:12:57.734 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 170 k 2024-08-19T13:12:57.734 INFO:teuthology.orchestra.run.smithi122.stdout: sqlite-devel x86_64 3.34.1-7.el9 appstream 134 k 2024-08-19T13:12:57.734 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrading: 2024-08-19T13:12:57.734 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.4 M 2024-08-19T13:12:57.734 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.2 M 2024-08-19T13:12:57.734 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-08-19T13:12:57.734 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-19T13:12:57.734 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-19T13:12:57.734 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 21 M 2024-08-19T13:12:57.735 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 32 k 2024-08-19T13:12:57.735 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 2.4 M 2024-08-19T13:12:57.735 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 249 k 2024-08-19T13:12:57.735 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 4.7 M 2024-08-19T13:12:57.735 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 17 M 2024-08-19T13:12:57.735 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 16 k 2024-08-19T13:12:57.735 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 25 k 2024-08-19T13:12:57.735 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-19T13:12:57.736 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-19T13:12:57.736 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-19T13:12:57.736 INFO:teuthology.orchestra.run.smithi122.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-19T13:12:57.736 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-19T13:12:57.736 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-19T13:12:57.736 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-19T13:12:57.736 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-19T13:12:57.736 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 163 k 2024-08-19T13:12:57.736 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-19T13:12:57.737 INFO:teuthology.orchestra.run.smithi122.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-19T13:12:57.737 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-19T13:12:57.737 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-19T13:12:57.737 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-19T13:12:57.737 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 507 k 2024-08-19T13:12:57.737 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-19T13:12:57.737 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T13:12:57.737 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-19T13:12:57.738 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-19T13:12:57.738 INFO:teuthology.orchestra.run.smithi122.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-19T13:12:57.738 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-19T13:12:57.738 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-19T13:12:57.738 INFO:teuthology.orchestra.run.smithi122.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-19T13:12:57.738 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-19T13:12:57.738 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-19T13:12:57.738 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-19T13:12:57.738 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-19T13:12:57.738 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-19T13:12:57.738 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-19T13:12:57.739 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-19T13:12:57.739 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-19T13:12:57.739 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 45 k 2024-08-19T13:12:57.739 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 136 k 2024-08-19T13:12:57.739 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-19T13:12:57.739 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-19T13:12:57.739 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-19T13:12:57.739 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-19T13:12:57.739 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-19T13:12:57.739 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel x86_64 3.9.19-4.el9 appstream 244 k 2024-08-19T13:12:57.739 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-19T13:12:57.739 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-19T13:12:57.740 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-19T13:12:57.740 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-19T13:12:57.740 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-19T13:12:57.740 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-19T13:12:57.740 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-19T13:12:57.740 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-19T13:12:57.740 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-19T13:12:57.740 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-19T13:12:57.740 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-19T13:12:57.740 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-19T13:12:57.740 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-19T13:12:57.741 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-19T13:12:57.741 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-19T13:12:57.741 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-19T13:12:57.741 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-19T13:12:57.741 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-19T13:12:57.741 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-19T13:12:57.741 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-19T13:12:57.741 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-19T13:12:57.741 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-19T13:12:57.742 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-19T13:12:57.742 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-19T13:12:57.742 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-19T13:12:57.742 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-19T13:12:57.742 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-19T13:12:57.742 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-19T13:12:57.742 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-19T13:12:57.742 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-19T13:12:57.742 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-19T13:12:57.742 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-19T13:12:57.742 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-19T13:12:57.743 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-19T13:12:57.743 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-19T13:12:57.743 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-19T13:12:57.743 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-19T13:12:57.743 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-19T13:12:57.743 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-19T13:12:57.743 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-19T13:12:57.743 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-19T13:12:57.743 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-19T13:12:57.743 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-19T13:12:57.743 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-19T13:12:57.743 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-19T13:12:57.743 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-19T13:12:57.744 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-19T13:12:57.744 INFO:teuthology.orchestra.run.smithi122.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-19T13:12:57.744 INFO:teuthology.orchestra.run.smithi122.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-19T13:12:57.744 INFO:teuthology.orchestra.run.smithi122.stdout: sqlite x86_64 3.34.1-7.el9 appstream 748 k 2024-08-19T13:12:57.744 INFO:teuthology.orchestra.run.smithi122.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-19T13:12:57.744 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-19T13:12:57.744 INFO:teuthology.orchestra.run.smithi122.stdout:Installing weak dependencies: 2024-08-19T13:12:57.744 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-19T13:12:57.744 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:12:57.744 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-08-19T13:12:57.744 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-08-19T13:12:57.744 INFO:teuthology.orchestra.run.smithi122.stdout:Install 126 Packages 2024-08-19T13:12:57.744 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrade 2 Packages 2024-08-19T13:12:57.745 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:12:57.745 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 202 M 2024-08-19T13:12:57.745 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-08-19T13:12:57.785 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-08-19T13:12:57.792 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-08-19T13:12:57.792 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-08-19T13:12:57.792 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-08-19T13:12:57.792 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-08-19T13:12:57.792 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 6.6 k 2024-08-19T13:12:57.792 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T13:12:57.792 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 896 k 2024-08-19T13:12:57.793 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 145 k 2024-08-19T13:12:57.793 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 1.5 M 2024-08-19T13:12:57.793 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 146 k 2024-08-19T13:12:57.793 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 3.6 M 2024-08-19T13:12:57.793 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 7.4 M 2024-08-19T13:12:57.793 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 49 k 2024-08-19T13:12:57.793 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 11 M 2024-08-19T13:12:57.793 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 49 M 2024-08-19T13:12:57.793 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 285 k 2024-08-19T13:12:57.793 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 766 k 2024-08-19T13:12:57.793 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 32 k 2024-08-19T13:12:57.793 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 753 k 2024-08-19T13:12:57.793 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 127 k 2024-08-19T13:12:57.794 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 188 k 2024-08-19T13:12:57.794 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 369 k 2024-08-19T13:12:57.794 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 350 k 2024-08-19T13:12:57.794 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 117 k 2024-08-19T13:12:57.794 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 85 k 2024-08-19T13:12:57.794 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.1 M 2024-08-19T13:12:57.794 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 170 k 2024-08-19T13:12:57.795 INFO:teuthology.orchestra.run.smithi103.stdout: sqlite-devel x86_64 3.34.1-7.el9 appstream 134 k 2024-08-19T13:12:57.795 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-08-19T13:12:57.795 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.4 M 2024-08-19T13:12:57.795 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.2 M 2024-08-19T13:12:57.795 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-08-19T13:12:57.795 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-19T13:12:57.795 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-19T13:12:57.795 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 21 M 2024-08-19T13:12:57.795 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 32 k 2024-08-19T13:12:57.795 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 2.4 M 2024-08-19T13:12:57.795 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 249 k 2024-08-19T13:12:57.796 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 4.7 M 2024-08-19T13:12:57.796 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 17 M 2024-08-19T13:12:57.796 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 16 k 2024-08-19T13:12:57.796 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 25 k 2024-08-19T13:12:57.796 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-19T13:12:57.796 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-19T13:12:57.796 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-19T13:12:57.796 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-19T13:12:57.796 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-19T13:12:57.796 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-19T13:12:57.796 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-19T13:12:57.796 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-19T13:12:57.796 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 163 k 2024-08-19T13:12:57.797 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-19T13:12:57.797 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-19T13:12:57.797 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-19T13:12:57.797 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-19T13:12:57.797 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-19T13:12:57.797 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 507 k 2024-08-19T13:12:57.797 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-19T13:12:57.797 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T13:12:57.797 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-19T13:12:57.797 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-19T13:12:57.797 INFO:teuthology.orchestra.run.smithi103.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-19T13:12:57.797 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-19T13:12:57.797 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-19T13:12:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-19T13:12:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-19T13:12:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-19T13:12:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-19T13:12:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-19T13:12:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-19T13:12:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-19T13:12:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-19T13:12:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-19T13:12:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 45 k 2024-08-19T13:12:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 136 k 2024-08-19T13:12:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-19T13:12:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-19T13:12:57.799 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-19T13:12:57.799 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-19T13:12:57.799 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-19T13:12:57.799 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.19-4.el9 appstream 244 k 2024-08-19T13:12:57.799 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-19T13:12:57.799 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-19T13:12:57.799 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-19T13:12:57.799 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-19T13:12:57.800 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-19T13:12:57.800 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-19T13:12:57.800 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-19T13:12:57.800 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-19T13:12:57.800 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-19T13:12:57.800 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-19T13:12:57.800 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-19T13:12:57.800 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-19T13:12:57.800 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-19T13:12:57.800 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-19T13:12:57.800 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-19T13:12:57.800 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-19T13:12:57.800 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-19T13:12:57.801 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-19T13:12:57.801 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-19T13:12:57.801 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-19T13:12:57.801 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-19T13:12:57.801 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-19T13:12:57.801 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-19T13:12:57.801 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-19T13:12:57.801 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-19T13:12:57.801 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-19T13:12:57.802 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-19T13:12:57.802 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-19T13:12:57.802 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-19T13:12:57.802 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-19T13:12:57.802 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-19T13:12:57.802 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-19T13:12:57.802 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-19T13:12:57.802 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-19T13:12:57.802 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-19T13:12:57.803 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-19T13:12:57.803 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-19T13:12:57.803 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-19T13:12:57.803 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-19T13:12:57.803 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-19T13:12:57.803 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-19T13:12:57.803 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-19T13:12:57.803 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-19T13:12:57.803 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-19T13:12:57.803 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-19T13:12:57.804 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-19T13:12:57.804 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-19T13:12:57.804 INFO:teuthology.orchestra.run.smithi103.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-19T13:12:57.804 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-19T13:12:57.804 INFO:teuthology.orchestra.run.smithi103.stdout: sqlite x86_64 3.34.1-7.el9 appstream 748 k 2024-08-19T13:12:57.804 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-19T13:12:57.804 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-19T13:12:57.804 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-08-19T13:12:57.804 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-19T13:12:57.804 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:12:57.804 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-08-19T13:12:57.804 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-08-19T13:12:57.804 INFO:teuthology.orchestra.run.smithi103.stdout:Install 126 Packages 2024-08-19T13:12:57.805 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 2 Packages 2024-08-19T13:12:57.805 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:12:57.805 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 202 M 2024-08-19T13:12:57.805 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-08-19T13:12:58.058 INFO:teuthology.orchestra.run.smithi122.stdout:(1/128): ceph-19.1.0-1382.gde09dd01.el9.x86_64. 55 kB/s | 6.6 kB 00:00 2024-08-19T13:12:58.242 INFO:teuthology.orchestra.run.smithi122.stdout:(2/128): ceph-fuse-19.1.0-1382.gde09dd01.el9.x8 4.8 MB/s | 896 kB 00:00 2024-08-19T13:12:58.317 INFO:teuthology.orchestra.run.smithi122.stdout:(3/128): ceph-immutable-object-cache-19.1.0-138 1.9 MB/s | 145 kB 00:00 2024-08-19T13:12:58.426 INFO:teuthology.orchestra.run.smithi122.stdout:(4/128): ceph-base-19.1.0-1382.gde09dd01.el9.x8 11 MB/s | 5.4 MB 00:00 2024-08-19T13:12:58.552 INFO:teuthology.orchestra.run.smithi122.stdout:(5/128): ceph-mds-19.1.0-1382.gde09dd01.el9.x86 10 MB/s | 2.4 MB 00:00 2024-08-19T13:12:58.585 INFO:teuthology.orchestra.run.smithi122.stdout:(6/128): ceph-mgr-19.1.0-1382.gde09dd01.el9.x86 9.6 MB/s | 1.5 MB 00:00 2024-08-19T13:12:58.871 INFO:teuthology.orchestra.run.smithi103.stdout:(1/128): ceph-19.1.0-1382.gde09dd01.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-08-19T13:12:59.097 INFO:teuthology.orchestra.run.smithi103.stdout:(2/128): ceph-fuse-19.1.0-1382.gde09dd01.el9.x8 3.9 MB/s | 896 kB 00:00 2024-08-19T13:12:59.147 INFO:teuthology.orchestra.run.smithi103.stdout:(3/128): ceph-immutable-object-cache-19.1.0-138 2.9 MB/s | 145 kB 00:00 2024-08-19T13:12:59.185 INFO:teuthology.orchestra.run.smithi122.stdout:(7/128): ceph-common-19.1.0-1382.gde09dd01.el9. 17 MB/s | 21 MB 00:01 2024-08-19T13:12:59.252 INFO:teuthology.orchestra.run.smithi122.stdout:(8/128): ceph-mon-19.1.0-1382.gde09dd01.el9.x86 6.7 MB/s | 4.7 MB 00:00 2024-08-19T13:12:59.289 INFO:teuthology.orchestra.run.smithi103.stdout:(4/128): ceph-base-19.1.0-1382.gde09dd01.el9.x8 7.6 MB/s | 5.4 MB 00:00 2024-08-19T13:12:59.302 INFO:teuthology.orchestra.run.smithi122.stdout:(9/128): ceph-selinux-19.1.0-1382.gde09dd01.el9 506 kB/s | 25 kB 00:00 2024-08-19T13:12:59.389 INFO:teuthology.orchestra.run.smithi103.stdout:(5/128): ceph-mds-19.1.0-1382.gde09dd01.el9.x86 9.9 MB/s | 2.4 MB 00:00 2024-08-19T13:12:59.448 INFO:teuthology.orchestra.run.smithi103.stdout:(6/128): ceph-mgr-19.1.0-1382.gde09dd01.el9.x86 9.6 MB/s | 1.5 MB 00:00 2024-08-19T13:12:59.619 INFO:teuthology.orchestra.run.smithi122.stdout:(10/128): ceph-osd-19.1.0-1382.gde09dd01.el9.x8 16 MB/s | 17 MB 00:01 2024-08-19T13:12:59.728 INFO:teuthology.orchestra.run.smithi122.stdout:(11/128): libcephfs-devel-19.1.0-1382.gde09dd01 294 kB/s | 32 kB 00:00 2024-08-19T13:12:59.995 INFO:teuthology.orchestra.run.smithi122.stdout:(12/128): libcephfs2-19.1.0-1382.gde09dd01.el9. 2.8 MB/s | 753 kB 00:00 2024-08-19T13:13:00.098 INFO:teuthology.orchestra.run.smithi122.stdout:(13/128): libcephsqlite-19.1.0-1382.gde09dd01.e 1.6 MB/s | 163 kB 00:00 2024-08-19T13:13:00.145 INFO:teuthology.orchestra.run.smithi122.stdout:(14/128): librados-devel-19.1.0-1382.gde09dd01. 2.5 MB/s | 127 kB 00:00 2024-08-19T13:13:00.223 INFO:teuthology.orchestra.run.smithi103.stdout:(7/128): ceph-osd-19.1.0-1382.gde09dd01.el9.x86 22 MB/s | 17 MB 00:00 2024-08-19T13:13:00.229 INFO:teuthology.orchestra.run.smithi122.stdout:(15/128): libradosstriper1-19.1.0-1382.gde09dd0 5.9 MB/s | 507 kB 00:00 2024-08-19T13:13:00.290 INFO:teuthology.orchestra.run.smithi103.stdout:(8/128): ceph-mon-19.1.0-1382.gde09dd01.el9.x86 5.2 MB/s | 4.7 MB 00:00 2024-08-19T13:13:00.357 INFO:teuthology.orchestra.run.smithi103.stdout:(9/128): ceph-selinux-19.1.0-1382.gde09dd01.el9 378 kB/s | 25 kB 00:00 2024-08-19T13:13:00.705 INFO:teuthology.orchestra.run.smithi122.stdout:(16/128): ceph-radosgw-19.1.0-1382.gde09dd01.el 6.9 MB/s | 11 MB 00:01 2024-08-19T13:13:00.763 INFO:teuthology.orchestra.run.smithi122.stdout:(17/128): python3-ceph-argparse-19.1.0-1382.gde 780 kB/s | 45 kB 00:00 2024-08-19T13:13:00.765 INFO:teuthology.orchestra.run.smithi103.stdout:(10/128): ceph-radosgw-19.1.0-1382.gde09dd01.el 19 MB/s | 11 MB 00:00 2024-08-19T13:13:00.832 INFO:teuthology.orchestra.run.smithi103.stdout:(11/128): libcephfs-devel-19.1.0-1382.gde09dd01 477 kB/s | 32 kB 00:00 2024-08-19T13:13:00.847 INFO:teuthology.orchestra.run.smithi122.stdout:(18/128): python3-ceph-common-19.1.0-1382.gde09 1.6 MB/s | 136 kB 00:00 2024-08-19T13:13:00.924 INFO:teuthology.orchestra.run.smithi103.stdout:(12/128): libcephfs2-19.1.0-1382.gde09dd01.el9. 8.0 MB/s | 753 kB 00:00 2024-08-19T13:13:00.947 INFO:teuthology.orchestra.run.smithi122.stdout:(19/128): python3-cephfs-19.1.0-1382.gde09dd01. 1.8 MB/s | 188 kB 00:00 2024-08-19T13:13:01.014 INFO:teuthology.orchestra.run.smithi122.stdout:(20/128): python3-rados-19.1.0-1382.gde09dd01.e 5.4 MB/s | 369 kB 00:00 2024-08-19T13:13:01.089 INFO:teuthology.orchestra.run.smithi122.stdout:(21/128): python3-rbd-19.1.0-1382.gde09dd01.el9 4.6 MB/s | 350 kB 00:00 2024-08-19T13:13:01.133 INFO:teuthology.orchestra.run.smithi103.stdout:(13/128): ceph-common-19.1.0-1382.gde09dd01.el9 8.3 MB/s | 21 MB 00:02 2024-08-19T13:13:01.139 INFO:teuthology.orchestra.run.smithi122.stdout:(22/128): python3-rgw-19.1.0-1382.gde09dd01.el9 2.3 MB/s | 117 kB 00:00 2024-08-19T13:13:01.158 INFO:teuthology.orchestra.run.smithi103.stdout:(14/128): libcephsqlite-19.1.0-1382.gde09dd01.e 699 kB/s | 163 kB 00:00 2024-08-19T13:13:01.184 INFO:teuthology.orchestra.run.smithi103.stdout:(15/128): librados-devel-19.1.0-1382.gde09dd01. 2.5 MB/s | 127 kB 00:00 2024-08-19T13:13:01.190 INFO:teuthology.orchestra.run.smithi122.stdout:(23/128): rbd-fuse-19.1.0-1382.gde09dd01.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-19T13:13:01.217 INFO:teuthology.orchestra.run.smithi103.stdout:(16/128): libradosstriper1-19.1.0-1382.gde09dd0 8.5 MB/s | 507 kB 00:00 2024-08-19T13:13:01.284 INFO:teuthology.orchestra.run.smithi103.stdout:(17/128): python3-ceph-argparse-19.1.0-1382.gde 674 kB/s | 45 kB 00:00 2024-08-19T13:13:01.360 INFO:teuthology.orchestra.run.smithi103.stdout:(18/128): python3-ceph-common-19.1.0-1382.gde09 1.8 MB/s | 136 kB 00:00 2024-08-19T13:13:01.435 INFO:teuthology.orchestra.run.smithi103.stdout:(19/128): python3-cephfs-19.1.0-1382.gde09dd01. 2.5 MB/s | 188 kB 00:00 2024-08-19T13:13:01.498 INFO:teuthology.orchestra.run.smithi122.stdout:(24/128): librgw2-19.1.0-1382.gde09dd01.el9.x86 4.2 MB/s | 5.4 MB 00:01 2024-08-19T13:13:01.502 INFO:teuthology.orchestra.run.smithi103.stdout:(20/128): python3-rados-19.1.0-1382.gde09dd01.e 5.4 MB/s | 369 kB 00:00 2024-08-19T13:13:01.548 INFO:teuthology.orchestra.run.smithi122.stdout:(25/128): rbd-nbd-19.1.0-1382.gde09dd01.el9.x86 3.4 MB/s | 170 kB 00:00 2024-08-19T13:13:01.591 INFO:teuthology.orchestra.run.smithi122.stdout:(26/128): ceph-grafana-dashboards-19.1.0-1382.g 766 kB/s | 32 kB 00:00 2024-08-19T13:13:01.619 INFO:teuthology.orchestra.run.smithi103.stdout:(21/128): librgw2-19.1.0-1382.gde09dd01.el9.x86 12 MB/s | 5.4 MB 00:00 2024-08-19T13:13:01.641 INFO:teuthology.orchestra.run.smithi122.stdout:(27/128): ceph-mgr-cephadm-19.1.0-1382.gde09dd0 2.9 MB/s | 146 kB 00:00 2024-08-19T13:13:01.644 INFO:teuthology.orchestra.run.smithi103.stdout:(22/128): python3-rbd-19.1.0-1382.gde09dd01.el9 2.4 MB/s | 350 kB 00:00 2024-08-19T13:13:01.678 INFO:teuthology.orchestra.run.smithi103.stdout:(23/128): python3-rgw-19.1.0-1382.gde09dd01.el9 1.9 MB/s | 117 kB 00:00 2024-08-19T13:13:01.703 INFO:teuthology.orchestra.run.smithi103.stdout:(24/128): rbd-fuse-19.1.0-1382.gde09dd01.el9.x8 1.4 MB/s | 85 kB 00:00 2024-08-19T13:13:01.716 INFO:teuthology.orchestra.run.smithi122.stdout:(28/128): rbd-mirror-19.1.0-1382.gde09dd01.el9. 5.9 MB/s | 3.1 MB 00:00 2024-08-19T13:13:01.770 INFO:teuthology.orchestra.run.smithi103.stdout:(25/128): rbd-nbd-19.1.0-1382.gde09dd01.el9.x86 2.5 MB/s | 170 kB 00:00 2024-08-19T13:13:01.830 INFO:teuthology.orchestra.run.smithi103.stdout:(26/128): ceph-grafana-dashboards-19.1.0-1382.g 548 kB/s | 32 kB 00:00 2024-08-19T13:13:01.904 INFO:teuthology.orchestra.run.smithi103.stdout:(27/128): rbd-mirror-19.1.0-1382.gde09dd01.el9. 14 MB/s | 3.1 MB 00:00 2024-08-19T13:13:01.929 INFO:teuthology.orchestra.run.smithi103.stdout:(28/128): ceph-mgr-cephadm-19.1.0-1382.gde09dd0 1.4 MB/s | 146 kB 00:00 2024-08-19T13:13:02.221 INFO:teuthology.orchestra.run.smithi103.stdout:(29/128): ceph-mgr-dashboard-19.1.0-1382.gde09d 11 MB/s | 3.6 MB 00:00 2024-08-19T13:13:02.297 INFO:teuthology.orchestra.run.smithi103.stdout:(30/128): ceph-mgr-modules-core-19.1.0-1382.gde 3.3 MB/s | 249 kB 00:00 2024-08-19T13:13:02.389 INFO:teuthology.orchestra.run.smithi103.stdout:(31/128): ceph-mgr-diskprediction-local-19.1.0- 16 MB/s | 7.4 MB 00:00 2024-08-19T13:13:02.399 INFO:teuthology.orchestra.run.smithi122.stdout:(29/128): ceph-mgr-dashboard-19.1.0-1382.gde09d 4.7 MB/s | 3.6 MB 00:00 2024-08-19T13:13:02.414 INFO:teuthology.orchestra.run.smithi103.stdout:(32/128): ceph-mgr-rook-19.1.0-1382.gde09dd01.e 421 kB/s | 49 kB 00:00 2024-08-19T13:13:02.439 INFO:teuthology.orchestra.run.smithi103.stdout:(33/128): ceph-prometheus-alerts-19.1.0-1382.gd 328 kB/s | 16 kB 00:00 2024-08-19T13:13:02.494 INFO:teuthology.orchestra.run.smithi103.stdout:(34/128): ceph-volume-19.1.0-1382.gde09dd01.el9 3.5 MB/s | 285 kB 00:00 2024-08-19T13:13:02.508 INFO:teuthology.orchestra.run.smithi122.stdout:(30/128): ceph-mgr-modules-core-19.1.0-1382.gde 2.3 MB/s | 249 kB 00:00 2024-08-19T13:13:02.532 INFO:teuthology.orchestra.run.smithi103.stdout:(35/128): cephadm-19.1.0-1382.gde09dd01.el9.noa 8.1 MB/s | 766 kB 00:00 2024-08-19T13:13:02.558 INFO:teuthology.orchestra.run.smithi122.stdout:(31/128): ceph-mgr-rook-19.1.0-1382.gde09dd01.e 992 kB/s | 49 kB 00:00 2024-08-19T13:13:02.609 INFO:teuthology.orchestra.run.smithi122.stdout:(32/128): ceph-prometheus-alerts-19.1.0-1382.gd 329 kB/s | 16 kB 00:00 2024-08-19T13:13:02.649 INFO:teuthology.orchestra.run.smithi103.stdout:(36/128): fuse-2.9.9-16.el9.x86_64.rpm 516 kB/s | 80 kB 00:00 2024-08-19T13:13:02.684 INFO:teuthology.orchestra.run.smithi122.stdout:(33/128): ceph-volume-19.1.0-1382.gde09dd01.el9 3.7 MB/s | 285 kB 00:00 2024-08-19T13:13:02.757 INFO:teuthology.orchestra.run.smithi103.stdout:(37/128): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-08-19T13:13:02.767 INFO:teuthology.orchestra.run.smithi122.stdout:(34/128): ceph-mgr-diskprediction-local-19.1.0- 7.0 MB/s | 7.4 MB 00:01 2024-08-19T13:13:02.842 INFO:teuthology.orchestra.run.smithi122.stdout:(35/128): cephadm-19.1.0-1382.gde09dd01.el9.noa 4.7 MB/s | 766 kB 00:00 2024-08-19T13:13:02.866 INFO:teuthology.orchestra.run.smithi103.stdout:(38/128): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-08-19T13:13:02.900 INFO:teuthology.orchestra.run.smithi103.stdout:(39/128): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-08-19T13:13:02.975 INFO:teuthology.orchestra.run.smithi103.stdout:(40/128): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-08-19T13:13:03.067 INFO:teuthology.orchestra.run.smithi103.stdout:(41/128): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-08-19T13:13:03.142 INFO:teuthology.orchestra.run.smithi103.stdout:(42/128): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-08-19T13:13:03.226 INFO:teuthology.orchestra.run.smithi103.stdout:(43/128): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-08-19T13:13:03.318 INFO:teuthology.orchestra.run.smithi103.stdout:(44/128): python3-cryptography-36.0.1-4.el9.x86 2.7 MB/s | 1.2 MB 00:00 2024-08-19T13:13:03.360 INFO:teuthology.orchestra.run.smithi103.stdout:(45/128): boost-program-options-1.75.0-8.el9.x8 801 kB/s | 107 kB 00:00 2024-08-19T13:13:03.427 INFO:teuthology.orchestra.run.smithi103.stdout:(46/128): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-08-19T13:13:03.468 INFO:teuthology.orchestra.run.smithi122.stdout:(36/128): ceph-test-19.1.0-1382.gde09dd01.el9.x 12 MB/s | 49 MB 00:04 2024-08-19T13:13:03.469 INFO:teuthology.orchestra.run.smithi103.stdout:(47/128): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-08-19T13:13:03.493 INFO:teuthology.orchestra.run.smithi122.stdout:(37/128): fuse-2.9.9-16.el9.x86_64.rpm 110 kB/s | 80 kB 00:00 2024-08-19T13:13:03.528 INFO:teuthology.orchestra.run.smithi103.stdout:(48/128): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-08-19T13:13:03.578 INFO:teuthology.orchestra.run.smithi103.stdout:(49/128): librabbitmq-0.11.0-7.el9.x86_64.rpm 908 kB/s | 45 kB 00:00 2024-08-19T13:13:03.603 INFO:teuthology.orchestra.run.smithi103.stdout:(50/128): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-08-19T13:13:03.637 INFO:teuthology.orchestra.run.smithi103.stdout:(51/128): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-08-19T13:13:03.671 INFO:teuthology.orchestra.run.smithi103.stdout:(52/128): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-08-19T13:13:03.696 INFO:teuthology.orchestra.run.smithi103.stdout:(53/128): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-08-19T13:13:03.721 INFO:teuthology.orchestra.run.smithi103.stdout:(54/128): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-08-19T13:13:03.813 INFO:teuthology.orchestra.run.smithi103.stdout:(55/128): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-08-19T13:13:03.844 INFO:teuthology.orchestra.run.smithi122.stdout:(38/128): python3-cffi-1.14.5-5.el9.x86_64.rpm 724 kB/s | 253 kB 00:00 2024-08-19T13:13:03.855 INFO:teuthology.orchestra.run.smithi103.stdout:(56/128): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-08-19T13:13:03.897 INFO:teuthology.orchestra.run.smithi103.stdout:(57/128): python3-devel-3.9.19-4.el9.x86_64.rpm 5.8 MB/s | 244 kB 00:00 2024-08-19T13:13:03.902 INFO:teuthology.orchestra.run.smithi122.stdout:(39/128): libgfortran-11.4.1-3.el9.x86_64.rpm 758 kB/s | 803 kB 00:01 2024-08-19T13:13:03.981 INFO:teuthology.orchestra.run.smithi103.stdout:(58/128): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-08-19T13:13:04.006 INFO:teuthology.orchestra.run.smithi103.stdout:(59/128): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-08-19T13:13:04.031 INFO:teuthology.orchestra.run.smithi103.stdout:(60/128): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-08-19T13:13:04.044 INFO:teuthology.orchestra.run.smithi122.stdout:(40/128): python3-ply-3.11-14.el9.noarch.rpm 752 kB/s | 106 kB 00:00 2024-08-19T13:13:04.056 INFO:teuthology.orchestra.run.smithi103.stdout:(61/128): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-08-19T13:13:04.070 INFO:teuthology.orchestra.run.smithi122.stdout:(41/128): libquadmath-11.4.1-3.el9.x86_64.rpm 325 kB/s | 195 kB 00:00 2024-08-19T13:13:04.082 INFO:teuthology.orchestra.run.smithi103.stdout:(62/128): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-08-19T13:13:04.107 INFO:teuthology.orchestra.run.smithi103.stdout:(63/128): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-08-19T13:13:04.149 INFO:teuthology.orchestra.run.smithi103.stdout:(64/128): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-08-19T13:13:04.153 INFO:teuthology.orchestra.run.smithi122.stdout:(42/128): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-08-19T13:13:04.170 INFO:teuthology.orchestra.run.smithi122.stdout:(43/128): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-08-19T13:13:04.204 INFO:teuthology.orchestra.run.smithi122.stdout:(44/128): python3-cryptography-36.0.1-4.el9.x86 3.4 MB/s | 1.2 MB 00:00 2024-08-19T13:13:04.262 INFO:teuthology.orchestra.run.smithi122.stdout:(45/128): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-08-19T13:13:04.313 INFO:teuthology.orchestra.run.smithi122.stdout:(46/128): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-08-19T13:13:04.338 INFO:teuthology.orchestra.run.smithi122.stdout:(47/128): boost-program-options-1.75.0-8.el9.x8 638 kB/s | 107 kB 00:00 2024-08-19T13:13:04.363 INFO:teuthology.orchestra.run.smithi122.stdout:(48/128): flexiblas-openblas-openmp-3.0.4-8.el9 361 kB/s | 18 kB 00:00 2024-08-19T13:13:04.414 INFO:teuthology.orchestra.run.smithi122.stdout:(49/128): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-08-19T13:13:04.439 INFO:teuthology.orchestra.run.smithi122.stdout:(50/128): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-08-19T13:13:04.572 INFO:teuthology.orchestra.run.smithi122.stdout:(51/128): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-08-19T13:13:04.625 INFO:teuthology.orchestra.run.smithi103.stdout:(65/128): ceph-test-19.1.0-1382.gde09dd01.el9.x 11 MB/s | 49 MB 00:04 2024-08-19T13:13:04.639 INFO:teuthology.orchestra.run.smithi122.stdout:(52/128): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-08-19T13:13:04.650 INFO:teuthology.orchestra.run.smithi103.stdout:(66/128): python3-oauthlib-3.1.1-5.el9.noarch.r 444 kB/s | 222 kB 00:00 2024-08-19T13:13:04.665 INFO:teuthology.orchestra.run.smithi122.stdout:(53/128): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-08-19T13:13:04.690 INFO:teuthology.orchestra.run.smithi122.stdout:(54/128): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-08-19T13:13:04.733 INFO:teuthology.orchestra.run.smithi103.stdout:(67/128): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.8 MB/s | 5.1 MB 00:00 2024-08-19T13:13:04.750 INFO:teuthology.orchestra.run.smithi103.stdout:(68/128): python3-prettytable-0.7.2-27.el9.noar 422 kB/s | 42 kB 00:00 2024-08-19T13:13:04.767 INFO:teuthology.orchestra.run.smithi103.stdout:(69/128): python3-packaging-20.9-5.el9.noarch.r 543 kB/s | 77 kB 00:00 2024-08-19T13:13:04.801 INFO:teuthology.orchestra.run.smithi103.stdout:(70/128): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-08-19T13:13:04.827 INFO:teuthology.orchestra.run.smithi103.stdout:(71/128): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-08-19T13:13:04.843 INFO:teuthology.orchestra.run.smithi103.stdout:(72/128): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-08-19T13:13:04.860 INFO:teuthology.orchestra.run.smithi103.stdout:(73/128): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-08-19T13:13:04.894 INFO:teuthology.orchestra.run.smithi103.stdout:(74/128): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-08-19T13:13:04.911 INFO:teuthology.orchestra.run.smithi103.stdout:(75/128): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-08-19T13:13:04.945 INFO:teuthology.orchestra.run.smithi103.stdout:(76/128): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-08-19T13:13:04.961 INFO:teuthology.orchestra.run.smithi103.stdout:(77/128): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-08-19T13:13:04.987 INFO:teuthology.orchestra.run.smithi103.stdout:(78/128): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-08-19T13:13:04.999 INFO:teuthology.orchestra.run.smithi122.stdout:(55/128): protobuf-3.14.0-13.el9.x86_64.rpm 3.3 MB/s | 1.0 MB 00:00 2024-08-19T13:13:05.012 INFO:teuthology.orchestra.run.smithi103.stdout:(79/128): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-08-19T13:13:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:(80/128): sqlite-3.34.1-7.el9.x86_64.rpm 11 MB/s | 748 kB 00:00 2024-08-19T13:13:05.079 INFO:teuthology.orchestra.run.smithi103.stdout:(81/128): sqlite-devel-3.34.1-7.el9.x86_64.rpm 2.0 MB/s | 134 kB 00:00 2024-08-19T13:13:05.105 INFO:teuthology.orchestra.run.smithi103.stdout:(82/128): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-08-19T13:13:05.207 INFO:teuthology.orchestra.run.smithi122.stdout:(56/128): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-08-19T13:13:05.307 INFO:teuthology.orchestra.run.smithi122.stdout:(57/128): python3-devel-3.9.19-4.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-08-19T13:13:05.330 INFO:teuthology.orchestra.run.smithi103.stdout:(83/128): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-08-19T13:13:05.391 INFO:teuthology.orchestra.run.smithi122.stdout:(58/128): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-08-19T13:13:05.405 INFO:teuthology.orchestra.run.smithi103.stdout:(84/128): abseil-cpp-20211102.0-3.el9.x86_64.rp 7.2 MB/s | 548 kB 00:00 2024-08-19T13:13:05.439 INFO:teuthology.orchestra.run.smithi103.stdout:(85/128): gperftools-libs-2.9.1-2.el9.x86_64.rp 9.1 MB/s | 309 kB 00:00 2024-08-19T13:13:05.441 INFO:teuthology.orchestra.run.smithi122.stdout:(59/128): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-08-19T13:13:05.472 INFO:teuthology.orchestra.run.smithi103.stdout:(86/128): grpc-data-1.46.7-10.el9.noarch.rpm 586 kB/s | 19 kB 00:00 2024-08-19T13:13:05.491 INFO:teuthology.orchestra.run.smithi122.stdout:(60/128): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-08-19T13:13:05.575 INFO:teuthology.orchestra.run.smithi122.stdout:(61/128): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-08-19T13:13:05.625 INFO:teuthology.orchestra.run.smithi122.stdout:(62/128): python3-markupsafe-1.1.1-12.el9.x86_6 697 kB/s | 35 kB 00:00 2024-08-19T13:13:05.656 INFO:teuthology.orchestra.run.smithi103.stdout:(87/128): libarrow-9.0.0-11.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-08-19T13:13:05.681 INFO:teuthology.orchestra.run.smithi103.stdout:(88/128): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-19T13:13:05.706 INFO:teuthology.orchestra.run.smithi103.stdout:(89/128): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-19T13:13:05.732 INFO:teuthology.orchestra.run.smithi103.stdout:(90/128): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-08-19T13:13:05.765 INFO:teuthology.orchestra.run.smithi103.stdout:(91/128): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-08-19T13:13:05.824 INFO:teuthology.orchestra.run.smithi103.stdout:(92/128): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-08-19T13:13:05.849 INFO:teuthology.orchestra.run.smithi103.stdout:(93/128): lua-devel-5.4.4-4.el9.x86_64.rpm 29 kB/s | 22 kB 00:00 2024-08-19T13:13:05.875 INFO:teuthology.orchestra.run.smithi122.stdout:(63/128): openblas-openmp-0.3.21-2.el9.x86_64.r 3.9 MB/s | 4.7 MB 00:01 2024-08-19T13:13:05.891 INFO:teuthology.orchestra.run.smithi103.stdout:(94/128): python3-asyncssh-2.13.2-5.el9.noarch. 8.2 MB/s | 548 kB 00:00 2024-08-19T13:13:05.917 INFO:teuthology.orchestra.run.smithi103.stdout:(95/128): python3-cachetools-4.2.4-1.el9.noarch 1.4 MB/s | 32 kB 00:00 2024-08-19T13:13:05.942 INFO:teuthology.orchestra.run.smithi103.stdout:(96/128): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 471 kB/s | 43 kB 00:00 2024-08-19T13:13:05.959 INFO:teuthology.orchestra.run.smithi122.stdout:(64/128): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-08-19T13:13:05.967 INFO:teuthology.orchestra.run.smithi103.stdout:(97/128): python3-certifi-2023.05.07-4.el9.noar 281 kB/s | 14 kB 00:00 2024-08-19T13:13:06.001 INFO:teuthology.orchestra.run.smithi103.stdout:(98/128): python3-cheroot-8.6.0-4.el9.noarch.rp 2.9 MB/s | 172 kB 00:00 2024-08-19T13:13:06.027 INFO:teuthology.orchestra.run.smithi103.stdout:(99/128): python3-cherrypy-18.6.1-2.el9.noarch. 6.0 MB/s | 358 kB 00:00 2024-08-19T13:13:06.042 INFO:teuthology.orchestra.run.smithi122.stdout:(65/128): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-08-19T13:13:06.053 INFO:teuthology.orchestra.run.smithi103.stdout:(100/128): python3-google-auth-2.33.0-1.el9.noa 4.2 MB/s | 220 kB 00:00 2024-08-19T13:13:06.085 INFO:teuthology.orchestra.run.smithi103.stdout:(101/128): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-08-19T13:13:06.093 INFO:teuthology.orchestra.run.smithi122.stdout:(66/128): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-19T13:13:06.111 INFO:teuthology.orchestra.run.smithi103.stdout:(102/128): python3-jaraco-8.2.1-3.el9.noarch.rp 426 kB/s | 11 kB 00:00 2024-08-19T13:13:06.136 INFO:teuthology.orchestra.run.smithi103.stdout:(103/128): python3-jaraco-classes-3.2.1-5.el9.n 716 kB/s | 18 kB 00:00 2024-08-19T13:13:06.143 INFO:teuthology.orchestra.run.smithi122.stdout:(67/128): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-08-19T13:13:06.162 INFO:teuthology.orchestra.run.smithi103.stdout:(104/128): python3-jaraco-collections-3.0.0-8.e 925 kB/s | 23 kB 00:00 2024-08-19T13:13:06.203 INFO:teuthology.orchestra.run.smithi103.stdout:(105/128): python3-jaraco-functools-3.5.0-2.el9 469 kB/s | 19 kB 00:00 2024-08-19T13:13:06.227 INFO:teuthology.orchestra.run.smithi122.stdout:(68/128): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-08-19T13:13:06.237 INFO:teuthology.orchestra.run.smithi103.stdout:(106/128): python3-jaraco-text-3.2.0-6.el9.noar 595 kB/s | 20 kB 00:00 2024-08-19T13:13:06.279 INFO:teuthology.orchestra.run.smithi103.stdout:(107/128): protobuf-compiler-3.14.0-13.el9.x86_ 735 kB/s | 863 kB 00:01 2024-08-19T13:13:06.285 INFO:teuthology.orchestra.run.smithi122.stdout:(69/128): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-08-19T13:13:06.337 INFO:teuthology.orchestra.run.smithi103.stdout:(108/128): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-08-19T13:13:06.363 INFO:teuthology.orchestra.run.smithi103.stdout:(109/128): python3-more-itertools-8.12.0-2.el9. 3.4 MB/s | 79 kB 00:00 2024-08-19T13:13:06.369 INFO:teuthology.orchestra.run.smithi122.stdout:(70/128): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-08-19T13:13:06.388 INFO:teuthology.orchestra.run.smithi103.stdout:(110/128): python3-logutils-0.3.5-21.el9.noarch 425 kB/s | 46 kB 00:00 2024-08-19T13:13:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:(111/128): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-08-19T13:13:06.444 INFO:teuthology.orchestra.run.smithi122.stdout:(71/128): python3-babel-2.9.1-2.el9.noarch.rpm 4.1 MB/s | 6.0 MB 00:01 2024-08-19T13:13:06.447 INFO:teuthology.orchestra.run.smithi103.stdout:(112/128): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-08-19T13:13:06.464 INFO:teuthology.orchestra.run.smithi103.stdout:(113/128): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-08-19T13:13:06.471 INFO:teuthology.orchestra.run.smithi122.stdout:(72/128): python3-pytz-2021.1-5.el9.noarch.rpm 498 kB/s | 51 kB 00:00 2024-08-19T13:13:06.481 INFO:teuthology.orchestra.run.smithi103.stdout:(114/128): python3-repoze-lru-0.7-16.el9.noarch 1.8 MB/s | 31 kB 00:00 2024-08-19T13:13:06.489 INFO:teuthology.orchestra.run.smithi122.stdout:(73/128): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 54 kB 00:00 2024-08-19T13:13:06.506 INFO:teuthology.orchestra.run.smithi103.stdout:(115/128): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-08-19T13:13:06.531 INFO:teuthology.orchestra.run.smithi122.stdout:(74/128): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-08-19T13:13:06.531 INFO:teuthology.orchestra.run.smithi103.stdout:(116/128): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-08-19T13:13:06.548 INFO:teuthology.orchestra.run.smithi103.stdout:(117/128): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-08-19T13:13:06.574 INFO:teuthology.orchestra.run.smithi103.stdout:(118/128): python3-tempora-5.0.0-2.el9.noarch.r 856 kB/s | 36 kB 00:00 2024-08-19T13:13:06.599 INFO:teuthology.orchestra.run.smithi103.stdout:(119/128): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-08-19T13:13:06.614 INFO:teuthology.orchestra.run.smithi122.stdout:(75/128): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-08-19T13:13:06.624 INFO:teuthology.orchestra.run.smithi103.stdout:(120/128): python3-typing-extensions-4.12.2-2.e 985 kB/s | 74 kB 00:00 2024-08-19T13:13:06.649 INFO:teuthology.orchestra.run.smithi103.stdout:(121/128): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-08-19T13:13:06.656 INFO:teuthology.orchestra.run.smithi122.stdout:(76/128): qatlib-service-24.02.0-1.el9.x86_64.r 861 kB/s | 36 kB 00:00 2024-08-19T13:13:06.675 INFO:teuthology.orchestra.run.smithi103.stdout:(122/128): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-08-19T13:13:06.700 INFO:teuthology.orchestra.run.smithi103.stdout:(123/128): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-08-19T13:13:06.725 INFO:teuthology.orchestra.run.smithi103.stdout:(124/128): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-19T13:13:06.740 INFO:teuthology.orchestra.run.smithi122.stdout:(77/128): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.6 MB/s | 5.1 MB 00:01 2024-08-19T13:13:06.751 INFO:teuthology.orchestra.run.smithi103.stdout:(125/128): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-19T13:13:06.765 INFO:teuthology.orchestra.run.smithi122.stdout:(78/128): qatzip-libs-1.2.0-1.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-08-19T13:13:06.801 INFO:teuthology.orchestra.run.smithi103.stdout:(126/128): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-19T13:13:06.832 INFO:teuthology.orchestra.run.smithi122.stdout:(79/128): socat-1.7.4.1-6.el9.x86_64.rpm 3.2 MB/s | 304 kB 00:00 2024-08-19T13:13:06.907 INFO:teuthology.orchestra.run.smithi122.stdout:(80/128): sqlite-devel-3.34.1-7.el9.x86_64.rpm 1.7 MB/s | 134 kB 00:00 2024-08-19T13:13:06.966 INFO:teuthology.orchestra.run.smithi122.stdout:(81/128): sqlite-3.34.1-7.el9.x86_64.rpm 3.6 MB/s | 748 kB 00:00 2024-08-19T13:13:06.991 INFO:teuthology.orchestra.run.smithi122.stdout:(82/128): xmlstarlet-1.6.1-20.el9.x86_64.rpm 762 kB/s | 64 kB 00:00 2024-08-19T13:13:07.109 INFO:teuthology.orchestra.run.smithi103.stdout:(127/128): librbd1-19.1.0-1382.gde09dd01.el9.x8 8.8 MB/s | 3.2 MB 00:00 2024-08-19T13:13:07.218 INFO:teuthology.orchestra.run.smithi103.stdout:(128/128): librados2-19.1.0-1382.gde09dd01.el9. 6.9 MB/s | 3.4 MB 00:00 2024-08-19T13:13:07.225 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-08-19T13:13:07.225 INFO:teuthology.orchestra.run.smithi103.stdout:Total 21 MB/s | 202 MB 00:09 2024-08-19T13:13:07.525 INFO:teuthology.orchestra.run.smithi122.stdout:(83/128): lua-devel-5.4.4-4.el9.x86_64.rpm 40 kB/s | 22 kB 00:00 2024-08-19T13:13:07.617 INFO:teuthology.orchestra.run.smithi122.stdout:(84/128): abseil-cpp-20211102.0-3.el9.x86_64.rp 5.9 MB/s | 548 kB 00:00 2024-08-19T13:13:07.659 INFO:teuthology.orchestra.run.smithi122.stdout:(85/128): gperftools-libs-2.9.1-2.el9.x86_64.rp 7.3 MB/s | 309 kB 00:00 2024-08-19T13:13:07.684 INFO:teuthology.orchestra.run.smithi122.stdout:(86/128): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-08-19T13:13:07.893 INFO:teuthology.orchestra.run.smithi122.stdout:(87/128): libarrow-9.0.0-11.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-08-19T13:13:07.918 INFO:teuthology.orchestra.run.smithi122.stdout:(88/128): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-19T13:13:07.952 INFO:teuthology.orchestra.run.smithi122.stdout:(89/128): protobuf-compiler-3.14.0-13.el9.x86_6 899 kB/s | 863 kB 00:00 2024-08-19T13:13:07.985 INFO:teuthology.orchestra.run.smithi122.stdout:(90/128): liboath-2.6.7-2.el9.x86_64.rpm 726 kB/s | 49 kB 00:00 2024-08-19T13:13:08.019 INFO:teuthology.orchestra.run.smithi122.stdout:(91/128): luarocks-3.9.2-5.el9.noarch.rpm 5.6 MB/s | 151 kB 00:00 2024-08-19T13:13:08.026 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-08-19T13:13:08.053 INFO:teuthology.orchestra.run.smithi122.stdout:(92/128): libunwind-1.6.2-1.el9.x86_64.rpm 670 kB/s | 67 kB 00:00 2024-08-19T13:13:08.112 INFO:teuthology.orchestra.run.smithi122.stdout:(93/128): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 839 kB 00:00 2024-08-19T13:13:08.129 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-08-19T13:13:08.129 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-08-19T13:13:08.145 INFO:teuthology.orchestra.run.smithi122.stdout:(94/128): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-08-19T13:13:08.170 INFO:teuthology.orchestra.run.smithi122.stdout:(95/128): python3-asyncssh-2.13.2-5.el9.noarch. 4.6 MB/s | 548 kB 00:00 2024-08-19T13:13:08.196 INFO:teuthology.orchestra.run.smithi122.stdout:(96/128): python3-cachetools-4.2.4-1.el9.noarch 640 kB/s | 32 kB 00:00 2024-08-19T13:13:08.221 INFO:teuthology.orchestra.run.smithi122.stdout:(97/128): python3-certifi-2023.05.07-4.el9.noar 280 kB/s | 14 kB 00:00 2024-08-19T13:13:08.255 INFO:teuthology.orchestra.run.smithi122.stdout:(98/128): python3-cheroot-8.6.0-4.el9.noarch.rp 2.8 MB/s | 172 kB 00:00 2024-08-19T13:13:08.297 INFO:teuthology.orchestra.run.smithi122.stdout:(99/128): python3-google-auth-2.33.0-1.el9.noar 5.2 MB/s | 220 kB 00:00 2024-08-19T13:13:08.322 INFO:teuthology.orchestra.run.smithi122.stdout:(100/128): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-08-19T13:13:08.356 INFO:teuthology.orchestra.run.smithi122.stdout:(101/128): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-08-19T13:13:08.381 INFO:teuthology.orchestra.run.smithi122.stdout:(102/128): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-08-19T13:13:08.406 INFO:teuthology.orchestra.run.smithi122.stdout:(103/128): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-08-19T13:13:08.432 INFO:teuthology.orchestra.run.smithi122.stdout:(104/128): python3-jaraco-collections-3.0.0-8.e 925 kB/s | 23 kB 00:00 2024-08-19T13:13:08.457 INFO:teuthology.orchestra.run.smithi122.stdout:(105/128): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-08-19T13:13:08.483 INFO:teuthology.orchestra.run.smithi122.stdout:(106/128): python3-jaraco-text-3.2.0-6.el9.noar 785 kB/s | 20 kB 00:00 2024-08-19T13:13:08.541 INFO:teuthology.orchestra.run.smithi122.stdout:(107/128): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-08-19T13:13:08.566 INFO:teuthology.orchestra.run.smithi122.stdout:(108/128): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-19T13:13:08.592 INFO:teuthology.orchestra.run.smithi122.stdout:(109/128): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-19T13:13:08.617 INFO:teuthology.orchestra.run.smithi122.stdout:(110/128): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-19T13:13:08.651 INFO:teuthology.orchestra.run.smithi122.stdout:(111/128): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-08-19T13:13:08.676 INFO:teuthology.orchestra.run.smithi122.stdout:(112/128): python3-portend-3.1.0-2.el9.noarch.r 657 kB/s | 16 kB 00:00 2024-08-19T13:13:08.702 INFO:teuthology.orchestra.run.smithi122.stdout:(113/128): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-08-19T13:13:08.727 INFO:teuthology.orchestra.run.smithi122.stdout:(114/128): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-08-19T13:13:08.744 INFO:teuthology.orchestra.run.smithi122.stdout:(115/128): python3-repoze-lru-0.7-16.el9.noarch 732 kB/s | 31 kB 00:00 2024-08-19T13:13:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:(116/128): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-08-19T13:13:08.786 INFO:teuthology.orchestra.run.smithi122.stdout:(117/128): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-08-19T13:13:08.803 INFO:teuthology.orchestra.run.smithi122.stdout:(118/128): python3-tempora-5.0.0-2.el9.noarch.r 855 kB/s | 36 kB 00:00 2024-08-19T13:13:08.829 INFO:teuthology.orchestra.run.smithi122.stdout:(119/128): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-08-19T13:13:08.854 INFO:teuthology.orchestra.run.smithi122.stdout:(120/128): python3-websocket-client-1.2.3-2.el9 3.6 MB/s | 90 kB 00:00 2024-08-19T13:13:08.879 INFO:teuthology.orchestra.run.smithi122.stdout:(121/128): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-08-19T13:13:08.904 INFO:teuthology.orchestra.run.smithi122.stdout:(122/128): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-08-19T13:13:08.930 INFO:teuthology.orchestra.run.smithi122.stdout:(123/128): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-08-19T13:13:08.955 INFO:teuthology.orchestra.run.smithi122.stdout:(124/128): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-19T13:13:09.022 INFO:teuthology.orchestra.run.smithi122.stdout:(125/128): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-08-19T13:13:09.248 INFO:teuthology.orchestra.run.smithi122.stdout:(126/128): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.9 MB/s | 16 MB 00:02 2024-08-19T13:13:09.373 INFO:teuthology.orchestra.run.smithi122.stdout:(127/128): librados2-19.1.0-1382.gde09dd01.el9. 8.2 MB/s | 3.4 MB 00:00 2024-08-19T13:13:09.598 INFO:teuthology.orchestra.run.smithi122.stdout:(128/128): librbd1-19.1.0-1382.gde09dd01.el9.x8 5.5 MB/s | 3.2 MB 00:00 2024-08-19T13:13:09.603 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-08-19T13:13:09.603 INFO:teuthology.orchestra.run.smithi122.stdout:Total 17 MB/s | 202 MB 00:11 2024-08-19T13:13:09.659 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-08-19T13:13:09.659 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-08-19T13:13:10.439 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-08-19T13:13:10.540 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-08-19T13:13:10.540 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-08-19T13:13:12.073 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-08-19T13:13:12.073 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-08-19T13:13:14.749 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-08-19T13:13:14.899 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/130 2024-08-19T13:13:14.946 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/130 2024-08-19T13:13:14.989 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/130 2024-08-19T13:13:15.327 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/130 2024-08-19T13:13:15.447 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/130 2024-08-19T13:13:15.568 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/130 2024-08-19T13:13:15.598 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/130 2024-08-19T13:13:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/130 2024-08-19T13:13:15.732 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 7/130 2024-08-19T13:13:15.761 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/130 2024-08-19T13:13:15.798 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/130 2024-08-19T13:13:15.826 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 10/130 2024-08-19T13:13:15.929 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 10/130 2024-08-19T13:13:15.952 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 11/130 2024-08-19T13:13:16.014 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 11/130 2024-08-19T13:13:16.037 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 12/130 2024-08-19T13:13:16.060 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 12/130 2024-08-19T13:13:16.317 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 13/130 2024-08-19T13:13:16.361 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 13/130 2024-08-19T13:13:16.475 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : re2-1:20211101-3.el9.x86_64 14/130 2024-08-19T13:13:16.568 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 15/130 2024-08-19T13:13:16.603 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 16/130 2024-08-19T13:13:16.638 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 17/130 2024-08-19T13:13:16.713 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 18/130 2024-08-19T13:13:16.774 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 19/130 2024-08-19T13:13:16.814 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 20/130 2024-08-19T13:13:16.857 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/130 2024-08-19T13:13:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 22/130 2024-08-19T13:13:16.917 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-5.4.4-4.el9.x86_64 23/130 2024-08-19T13:13:16.988 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 24/130 2024-08-19T13:13:17.040 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 25/130 2024-08-19T13:13:17.070 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 26/130 2024-08-19T13:13:17.114 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 27/130 2024-08-19T13:13:17.125 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-08-19T13:13:17.191 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 28/130 2024-08-19T13:13:17.225 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 29/130 2024-08-19T13:13:17.259 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/130 2024-08-19T13:13:17.270 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 30/130 2024-08-19T13:13:17.297 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 31/130 2024-08-19T13:13:17.387 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 32/130 2024-08-19T13:13:17.445 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : luarocks-3.9.2-5.el9.noarch 33/130 2024-08-19T13:13:17.460 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/130 2024-08-19T13:13:17.505 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/130 2024-08-19T13:13:17.522 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 34/130 2024-08-19T13:13:17.658 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 35/130 2024-08-19T13:13:17.714 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 36/130 2024-08-19T13:13:17.834 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/130 2024-08-19T13:13:17.944 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/130 2024-08-19T13:13:18.067 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/130 2024-08-19T13:13:18.096 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/130 2024-08-19T13:13:18.191 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/130 2024-08-19T13:13:18.229 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 7/130 2024-08-19T13:13:18.258 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 37/130 2024-08-19T13:13:18.258 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/130 2024-08-19T13:13:18.295 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/130 2024-08-19T13:13:18.315 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 10/130 2024-08-19T13:13:18.345 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 38/130 2024-08-19T13:13:18.380 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 39/130 2024-08-19T13:13:18.411 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 10/130 2024-08-19T13:13:18.423 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 40/130 2024-08-19T13:13:18.433 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 11/130 2024-08-19T13:13:18.461 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 41/130 2024-08-19T13:13:18.479 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 11/130 2024-08-19T13:13:18.494 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 42/130 2024-08-19T13:13:18.500 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 12/130 2024-08-19T13:13:18.522 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 12/130 2024-08-19T13:13:18.550 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 43/130 2024-08-19T13:13:18.624 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 44/130 2024-08-19T13:13:18.662 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 45/130 2024-08-19T13:13:18.699 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 46/130 2024-08-19T13:13:18.746 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 47/130 2024-08-19T13:13:18.799 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 48/130 2024-08-19T13:13:18.842 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 49/130 2024-08-19T13:13:18.842 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 13/130 2024-08-19T13:13:18.876 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 13/130 2024-08-19T13:13:18.980 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 50/130 2024-08-19T13:13:18.985 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : re2-1:20211101-3.el9.x86_64 14/130 2024-08-19T13:13:19.027 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 51/130 2024-08-19T13:13:19.070 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 52/130 2024-08-19T13:13:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 15/130 2024-08-19T13:13:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 16/130 2024-08-19T13:13:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : liboath-2.6.7-2.el9.x86_64 17/130 2024-08-19T13:13:19.173 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 53/130 2024-08-19T13:13:19.209 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 54/130 2024-08-19T13:13:19.227 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 18/130 2024-08-19T13:13:19.249 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 55/130 2024-08-19T13:13:19.275 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 56/130 2024-08-19T13:13:19.300 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 57/130 2024-08-19T13:13:19.345 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 19/130 2024-08-19T13:13:19.376 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 58/130 2024-08-19T13:13:19.394 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-packaging-20.9-5.el9.noarch 20/130 2024-08-19T13:13:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/130 2024-08-19T13:13:19.470 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 22/130 2024-08-19T13:13:19.498 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-5.4.4-4.el9.x86_64 23/130 2024-08-19T13:13:19.512 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 59/130 2024-08-19T13:13:19.552 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 60/130 2024-08-19T13:13:19.568 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 24/130 2024-08-19T13:13:19.588 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 61/130 2024-08-19T13:13:19.629 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 25/130 2024-08-19T13:13:19.630 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 62/130 2024-08-19T13:13:19.659 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 26/130 2024-08-19T13:13:19.665 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 63/130 2024-08-19T13:13:19.695 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 64/130 2024-08-19T13:13:19.703 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 27/130 2024-08-19T13:13:19.729 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 65/130 2024-08-19T13:13:19.763 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 66/130 2024-08-19T13:13:19.781 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 28/130 2024-08-19T13:13:19.804 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 67/130 2024-08-19T13:13:19.814 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 29/130 2024-08-19T13:13:19.839 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 68/130 2024-08-19T13:13:19.852 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 30/130 2024-08-19T13:13:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 31/130 2024-08-19T13:13:19.927 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 69/130 2024-08-19T13:13:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 32/130 2024-08-19T13:13:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : luarocks-3.9.2-5.el9.noarch 33/130 2024-08-19T13:13:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 34/130 2024-08-19T13:13:20.168 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 70/130 2024-08-19T13:13:20.201 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/130 2024-08-19T13:13:20.202 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 35/130 2024-08-19T13:13:20.254 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 36/130 2024-08-19T13:13:20.309 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 72/130 2024-08-19T13:13:20.360 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 73/130 2024-08-19T13:13:20.775 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rsa-4.9-2.el9.noarch 37/130 2024-08-19T13:13:20.859 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 38/130 2024-08-19T13:13:20.887 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 39/130 2024-08-19T13:13:20.922 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 40/130 2024-08-19T13:13:20.960 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 41/130 2024-08-19T13:13:20.994 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 42/130 2024-08-19T13:13:21.049 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 43/130 2024-08-19T13:13:21.112 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/130 2024-08-19T13:13:21.121 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 44/130 2024-08-19T13:13:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 45/130 2024-08-19T13:13:21.166 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 75/130 2024-08-19T13:13:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 46/130 2024-08-19T13:13:21.199 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 76/130 2024-08-19T13:13:21.244 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 47/130 2024-08-19T13:13:21.289 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 48/130 2024-08-19T13:13:21.304 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 77/130 2024-08-19T13:13:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 49/130 2024-08-19T13:13:21.350 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.19-4.el9.x86_64 78/130 2024-08-19T13:13:21.468 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 50/130 2024-08-19T13:13:21.509 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 51/130 2024-08-19T13:13:21.529 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.21-2.el9.x86_64 79/130 2024-08-19T13:13:21.550 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 52/130 2024-08-19T13:13:21.556 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 80/130 2024-08-19T13:13:21.633 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 81/130 2024-08-19T13:13:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 53/130 2024-08-19T13:13:21.700 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 54/130 2024-08-19T13:13:21.740 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 55/130 2024-08-19T13:13:21.766 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 56/130 2024-08-19T13:13:21.791 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 57/130 2024-08-19T13:13:21.867 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 58/130 2024-08-19T13:13:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 59/130 2024-08-19T13:13:22.026 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 60/130 2024-08-19T13:13:22.063 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 61/130 2024-08-19T13:13:22.104 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 62/130 2024-08-19T13:13:22.139 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 63/130 2024-08-19T13:13:22.170 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 64/130 2024-08-19T13:13:22.206 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 65/130 2024-08-19T13:13:22.237 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 66/130 2024-08-19T13:13:22.279 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 67/130 2024-08-19T13:13:22.292 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 82/130 2024-08-19T13:13:22.314 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 68/130 2024-08-19T13:13:22.400 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 69/130 2024-08-19T13:13:22.424 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 83/130 2024-08-19T13:13:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 70/130 2024-08-19T13:13:22.484 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/130 2024-08-19T13:13:22.587 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 72/130 2024-08-19T13:13:22.635 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 73/130 2024-08-19T13:13:23.365 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/130 2024-08-19T13:13:23.416 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 75/130 2024-08-19T13:13:23.450 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 76/130 2024-08-19T13:13:23.551 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 77/130 2024-08-19T13:13:23.584 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-devel-3.9.19-4.el9.x86_64 78/130 2024-08-19T13:13:23.686 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/130 2024-08-19T13:13:23.759 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-0.3.21-2.el9.x86_64 79/130 2024-08-19T13:13:23.773 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 85/130 2024-08-19T13:13:23.798 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 80/130 2024-08-19T13:13:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 81/130 2024-08-19T13:13:24.082 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 86/130 2024-08-19T13:13:24.127 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 87/130 2024-08-19T13:13:24.221 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 87/130 2024-08-19T13:13:24.248 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 88/130 2024-08-19T13:13:24.284 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 89/130 2024-08-19T13:13:24.531 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 82/130 2024-08-19T13:13:24.666 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 83/130 2024-08-19T13:13:24.760 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 90/130 2024-08-19T13:13:24.788 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 91/130 2024-08-19T13:13:24.843 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 91/130 2024-08-19T13:13:24.873 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 92/130 2024-08-19T13:13:25.900 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/130 2024-08-19T13:13:25.984 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 85/130 2024-08-19T13:13:26.291 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 86/130 2024-08-19T13:13:26.338 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 87/130 2024-08-19T13:13:26.432 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 87/130 2024-08-19T13:13:26.460 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 88/130 2024-08-19T13:13:26.495 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 89/130 2024-08-19T13:13:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 90/130 2024-08-19T13:13:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 91/130 2024-08-19T13:13:27.055 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 91/130 2024-08-19T13:13:27.076 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 92/130 2024-08-19T13:13:27.120 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 93/130 2024-08-19T13:13:27.163 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 93/130 2024-08-19T13:13:27.735 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 93/130 2024-08-19T13:13:27.774 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 94/130 2024-08-19T13:13:27.870 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 94/130 2024-08-19T13:13:27.871 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-19T13:13:27.871 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-19T13:13:27.871 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:13:27.900 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 95/130 2024-08-19T13:13:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 93/130 2024-08-19T13:13:29.431 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 93/130 2024-08-19T13:13:29.980 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 93/130 2024-08-19T13:13:30.027 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 94/130 2024-08-19T13:13:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 94/130 2024-08-19T13:13:30.120 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-19T13:13:30.120 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-19T13:13:30.120 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:13:30.145 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 95/130 2024-08-19T13:13:39.192 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 95/130 2024-08-19T13:13:39.192 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-08-19T13:13:39.193 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-08-19T13:13:39.193 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-08-19T13:13:39.193 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-08-19T13:13:39.193 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-08-19T13:13:39.193 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-08-19T13:13:39.193 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-08-19T13:13:39.193 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:13:39.372 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 96/130 2024-08-19T13:13:39.388 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 96/130 2024-08-19T13:13:39.388 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:39.388 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-19T13:13:39.388 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:13:40.884 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 97/130 2024-08-19T13:13:40.910 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 97/130 2024-08-19T13:13:40.910 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:40.910 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-19T13:13:40.910 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T13:13:40.911 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T13:13:40.911 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:13:41.134 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 98/130 2024-08-19T13:13:41.157 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 98/130 2024-08-19T13:13:41.157 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:41.157 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-19T13:13:41.157 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T13:13:41.158 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T13:13:41.158 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:13:41.563 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 95/130 2024-08-19T13:13:41.563 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /sys 2024-08-19T13:13:41.563 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /proc 2024-08-19T13:13:41.563 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /mnt 2024-08-19T13:13:41.563 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /var/tmp 2024-08-19T13:13:41.563 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /home 2024-08-19T13:13:41.564 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /root 2024-08-19T13:13:41.564 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /tmp 2024-08-19T13:13:41.564 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:13:41.594 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 99/130 2024-08-19T13:13:41.616 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 99/130 2024-08-19T13:13:41.616 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:41.617 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-19T13:13:41.617 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T13:13:41.617 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T13:13:41.617 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:13:41.679 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 100/130 2024-08-19T13:13:41.735 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 101/130 2024-08-19T13:13:41.741 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 96/130 2024-08-19T13:13:41.758 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 96/130 2024-08-19T13:13:41.758 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:41.758 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-19T13:13:41.758 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:13:41.931 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 102/130 2024-08-19T13:13:41.985 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 103/130 2024-08-19T13:13:42.063 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 104/130 2024-08-19T13:13:42.151 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 105/130 2024-08-19T13:13:42.283 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 106/130 2024-08-19T13:13:42.327 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 107/130 2024-08-19T13:13:42.368 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 108/130 2024-08-19T13:13:42.395 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-16.el9.x86_64 109/130 2024-08-19T13:13:42.799 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 110/130 2024-08-19T13:13:42.828 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 110/130 2024-08-19T13:13:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 111/130 2024-08-19T13:13:42.921 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 112/130 2024-08-19T13:13:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 113/130 2024-08-19T13:13:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 97/130 2024-08-19T13:13:43.267 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 97/130 2024-08-19T13:13:43.267 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:43.267 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-19T13:13:43.267 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T13:13:43.267 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T13:13:43.267 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:13:43.499 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 98/130 2024-08-19T13:13:43.526 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 98/130 2024-08-19T13:13:43.526 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:43.526 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-19T13:13:43.526 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T13:13:43.527 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T13:13:43.527 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:13:43.958 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 99/130 2024-08-19T13:13:43.981 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 99/130 2024-08-19T13:13:43.982 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:43.982 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-19T13:13:43.982 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T13:13:43.982 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T13:13:43.982 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:13:44.001 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 113/130 2024-08-19T13:13:44.036 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 114/130 2024-08-19T13:13:44.043 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ply-3.11-14.el9.noarch 100/130 2024-08-19T13:13:44.099 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 101/130 2024-08-19T13:13:44.284 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 102/130 2024-08-19T13:13:44.333 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 103/130 2024-08-19T13:13:44.403 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 104/130 2024-08-19T13:13:44.490 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 105/130 2024-08-19T13:13:44.637 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 106/130 2024-08-19T13:13:44.675 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 107/130 2024-08-19T13:13:44.707 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 108/130 2024-08-19T13:13:44.736 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-2.9.9-16.el9.x86_64 109/130 2024-08-19T13:13:45.002 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 114/130 2024-08-19T13:13:45.028 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 115/130 2024-08-19T13:13:45.131 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 115/130 2024-08-19T13:13:45.206 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 110/130 2024-08-19T13:13:45.260 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 110/130 2024-08-19T13:13:45.291 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 111/130 2024-08-19T13:13:45.299 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 116/130 2024-08-19T13:13:45.322 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 117/130 2024-08-19T13:13:45.345 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 117/130 2024-08-19T13:13:45.345 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:45.346 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-19T13:13:45.346 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T13:13:45.346 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T13:13:45.346 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:13:45.353 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 112/130 2024-08-19T13:13:45.383 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 113/130 2024-08-19T13:13:45.389 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 118/130 2024-08-19T13:13:45.400 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 118/130 2024-08-19T13:13:45.498 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 119/130 2024-08-19T13:13:46.414 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 113/130 2024-08-19T13:13:46.417 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 120/130 2024-08-19T13:13:46.458 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 121/130 2024-08-19T13:13:46.459 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 114/130 2024-08-19T13:13:46.481 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 121/130 2024-08-19T13:13:46.481 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:46.481 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-19T13:13:46.481 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T13:13:46.481 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T13:13:46.481 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:13:46.525 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 122/130 2024-08-19T13:13:46.547 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1382.gde09d 122/130 2024-08-19T13:13:46.547 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:46.548 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-19T13:13:46.548 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:13:46.851 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 123/130 2024-08-19T13:13:46.874 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 123/130 2024-08-19T13:13:46.874 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:46.874 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-19T13:13:46.874 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T13:13:46.874 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T13:13:46.874 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:13:47.418 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 114/130 2024-08-19T13:13:47.452 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 115/130 2024-08-19T13:13:47.552 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 115/130 2024-08-19T13:13:47.723 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 116/130 2024-08-19T13:13:47.754 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 117/130 2024-08-19T13:13:47.778 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 117/130 2024-08-19T13:13:47.778 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:47.778 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-19T13:13:47.778 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T13:13:47.778 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T13:13:47.779 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:13:47.822 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 118/130 2024-08-19T13:13:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 118/130 2024-08-19T13:13:47.932 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 119/130 2024-08-19T13:13:48.836 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 120/130 2024-08-19T13:13:48.865 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 121/130 2024-08-19T13:13:48.889 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 121/130 2024-08-19T13:13:48.890 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:48.890 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-19T13:13:48.890 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T13:13:48.890 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T13:13:48.890 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:13:48.933 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 122/130 2024-08-19T13:13:48.955 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1382.gde09d 122/130 2024-08-19T13:13:48.956 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:48.956 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-19T13:13:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:13:49.259 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 123/130 2024-08-19T13:13:49.282 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 123/130 2024-08-19T13:13:49.283 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T13:13:49.283 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-19T13:13:49.283 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T13:13:49.283 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T13:13:49.283 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:13:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 124/130 2024-08-19T13:13:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 125/130 2024-08-19T13:13:51.477 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 126/130 2024-08-19T13:13:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : sqlite-devel-3.34.1-7.el9.x86_64 127/130 2024-08-19T13:13:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 128/130 2024-08-19T13:13:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 129/130 2024-08-19T13:13:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 129/130 2024-08-19T13:13:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 130/130 2024-08-19T13:13:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 124/130 2024-08-19T13:13:53.871 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 125/130 2024-08-19T13:13:53.902 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 126/130 2024-08-19T13:13:53.928 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : sqlite-devel-3.34.1-7.el9.x86_64 127/130 2024-08-19T13:13:53.959 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 128/130 2024-08-19T13:13:53.959 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 129/130 2024-08-19T13:13:54.035 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 129/130 2024-08-19T13:13:54.035 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 130/130 2024-08-19T13:13:56.863 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 130/130 2024-08-19T13:13:56.863 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 1/130 2024-08-19T13:13:56.864 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2/130 2024-08-19T13:13:56.864 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 3/130 2024-08-19T13:13:56.864 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 4/130 2024-08-19T13:13:56.864 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 5/130 2024-08-19T13:13:56.864 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 6/130 2024-08-19T13:13:56.864 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 7/130 2024-08-19T13:13:56.864 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 8/130 2024-08-19T13:13:56.864 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 9/130 2024-08-19T13:13:56.864 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 10/130 2024-08-19T13:13:56.864 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 11/130 2024-08-19T13:13:56.864 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 12/130 2024-08-19T13:13:56.864 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 13/130 2024-08-19T13:13:56.864 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 14/130 2024-08-19T13:13:56.865 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 15/130 2024-08-19T13:13:56.865 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 16/130 2024-08-19T13:13:56.865 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 17/130 2024-08-19T13:13:56.865 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 18/130 2024-08-19T13:13:56.865 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 19/130 2024-08-19T13:13:56.865 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 20/130 2024-08-19T13:13:56.865 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 21/130 2024-08-19T13:13:56.865 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 22/130 2024-08-19T13:13:56.865 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 23/130 2024-08-19T13:13:56.865 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 24/130 2024-08-19T13:13:56.865 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 25/130 2024-08-19T13:13:56.865 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 26/130 2024-08-19T13:13:56.866 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 27/130 2024-08-19T13:13:56.866 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 28/130 2024-08-19T13:13:56.866 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 29/130 2024-08-19T13:13:56.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 30/130 2024-08-19T13:13:56.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 31/130 2024-08-19T13:13:56.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 32/130 2024-08-19T13:13:56.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 33/130 2024-08-19T13:13:56.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 34/130 2024-08-19T13:13:56.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 35/130 2024-08-19T13:13:56.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 36/130 2024-08-19T13:13:56.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 37/130 2024-08-19T13:13:56.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 38/130 2024-08-19T13:13:56.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 39/130 2024-08-19T13:13:56.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/130 2024-08-19T13:13:56.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/130 2024-08-19T13:13:56.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/130 2024-08-19T13:13:56.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/130 2024-08-19T13:13:56.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/130 2024-08-19T13:13:56.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 45/130 2024-08-19T13:13:56.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 46/130 2024-08-19T13:13:56.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 47/130 2024-08-19T13:13:56.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 48/130 2024-08-19T13:13:56.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 49/130 2024-08-19T13:13:56.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 50/130 2024-08-19T13:13:56.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 51/130 2024-08-19T13:13:56.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 52/130 2024-08-19T13:13:56.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/130 2024-08-19T13:13:56.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/130 2024-08-19T13:13:56.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 55/130 2024-08-19T13:13:56.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 56/130 2024-08-19T13:13:56.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 57/130 2024-08-19T13:13:56.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/130 2024-08-19T13:13:56.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.19-4.el9.x86_64 59/130 2024-08-19T13:13:56.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 60/130 2024-08-19T13:13:56.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/130 2024-08-19T13:13:56.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/130 2024-08-19T13:13:56.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/130 2024-08-19T13:13:56.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/130 2024-08-19T13:13:56.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/130 2024-08-19T13:13:56.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/130 2024-08-19T13:13:56.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/130 2024-08-19T13:13:56.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/130 2024-08-19T13:13:56.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/130 2024-08-19T13:13:56.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/130 2024-08-19T13:13:56.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/130 2024-08-19T13:13:56.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/130 2024-08-19T13:13:56.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/130 2024-08-19T13:13:56.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/130 2024-08-19T13:13:56.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/130 2024-08-19T13:13:56.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/130 2024-08-19T13:13:56.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 77/130 2024-08-19T13:13:56.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 78/130 2024-08-19T13:13:56.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 79/130 2024-08-19T13:13:56.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 80/130 2024-08-19T13:13:56.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 81/130 2024-08-19T13:13:56.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : sqlite-devel-3.34.1-7.el9.x86_64 82/130 2024-08-19T13:13:56.879 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/130 2024-08-19T13:13:56.879 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 84/130 2024-08-19T13:13:56.879 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 85/130 2024-08-19T13:13:56.879 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 86/130 2024-08-19T13:13:56.879 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 87/130 2024-08-19T13:13:56.879 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 88/130 2024-08-19T13:13:56.879 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 89/130 2024-08-19T13:13:56.879 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 90/130 2024-08-19T13:13:56.879 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 91/130 2024-08-19T13:13:56.879 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 92/130 2024-08-19T13:13:56.879 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 93/130 2024-08-19T13:13:56.879 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 94/130 2024-08-19T13:13:56.879 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 95/130 2024-08-19T13:13:56.879 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 96/130 2024-08-19T13:13:56.880 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 97/130 2024-08-19T13:13:56.880 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 98/130 2024-08-19T13:13:56.880 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 99/130 2024-08-19T13:13:56.880 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 100/130 2024-08-19T13:13:56.880 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 101/130 2024-08-19T13:13:56.880 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 102/130 2024-08-19T13:13:56.880 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/130 2024-08-19T13:13:56.880 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 104/130 2024-08-19T13:13:56.880 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 105/130 2024-08-19T13:13:56.880 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 106/130 2024-08-19T13:13:56.880 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 107/130 2024-08-19T13:13:56.880 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 108/130 2024-08-19T13:13:56.880 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 109/130 2024-08-19T13:13:56.881 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 110/130 2024-08-19T13:13:56.881 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 111/130 2024-08-19T13:13:56.881 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 112/130 2024-08-19T13:13:56.881 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 113/130 2024-08-19T13:13:56.881 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 114/130 2024-08-19T13:13:56.881 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/130 2024-08-19T13:13:56.881 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 116/130 2024-08-19T13:13:56.881 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 117/130 2024-08-19T13:13:56.881 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 118/130 2024-08-19T13:13:56.881 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 119/130 2024-08-19T13:13:56.881 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 120/130 2024-08-19T13:13:56.881 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 121/130 2024-08-19T13:13:56.881 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 122/130 2024-08-19T13:13:56.881 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 123/130 2024-08-19T13:13:56.882 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 124/130 2024-08-19T13:13:56.882 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : re2-1:20211101-3.el9.x86_64 125/130 2024-08-19T13:13:56.882 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 126/130 2024-08-19T13:13:56.882 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 127/130 2024-08-19T13:13:56.882 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 128/130 2024-08-19T13:13:56.882 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 129/130 2024-08-19T13:13:58.802 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 130/130 2024-08-19T13:13:58.802 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:13:58.802 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-08-19T13:13:58.802 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.803 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.803 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-08-19T13:13:58.803 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-19T13:13:58.803 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-19T13:13:58.803 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.803 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.803 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.803 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.804 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:13:58.804 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.804 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.804 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.804 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:13:58.804 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:13:58.804 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:13:58.804 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:13:58.805 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:13:58.805 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.805 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.805 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:13:58.805 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.805 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.806 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.806 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:13:58.806 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:13:58.806 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-19T13:13:58.806 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-19T13:13:58.806 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-19T13:13:58.807 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-19T13:13:58.807 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-19T13:13:58.807 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-19T13:13:58.807 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-19T13:13:58.807 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-19T13:13:58.807 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.807 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.808 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.808 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-19T13:13:58.808 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-19T13:13:58.808 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-19T13:13:58.808 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-19T13:13:58.808 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-19T13:13:58.808 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.808 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.809 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-19T13:13:58.809 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.809 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-19T13:13:58.809 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-19T13:13:58.809 INFO:teuthology.orchestra.run.smithi103.stdout: lua-5.4.4-4.el9.x86_64 2024-08-19T13:13:58.809 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-19T13:13:58.809 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-19T13:13:58.809 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-19T13:13:58.809 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-19T13:13:58.809 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-19T13:13:58.809 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-19T13:13:58.809 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-19T13:13:58.809 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-19T13:13:58.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-19T13:13:58.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-19T13:13:58.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-19T13:13:58.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-19T13:13:58.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-19T13:13:58.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-19T13:13:58.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-19T13:13:58.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-19T13:13:58.811 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.19-4.el9.x86_64 2024-08-19T13:13:58.811 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-19T13:13:58.812 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-19T13:13:58.812 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-19T13:13:58.812 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-19T13:13:58.812 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-19T13:13:58.812 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-19T13:13:58.812 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-19T13:13:58.812 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-19T13:13:58.812 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-19T13:13:58.812 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-19T13:13:58.812 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-19T13:13:58.812 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-19T13:13:58.812 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-19T13:13:58.812 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-19T13:13:58.813 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-19T13:13:58.813 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-19T13:13:58.813 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-19T13:13:58.813 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-19T13:13:58.813 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-19T13:13:58.813 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-19T13:13:58.813 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-19T13:13:58.813 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-19T13:13:58.813 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-08-19T13:13:58.813 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-19T13:13:58.813 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-19T13:13:58.813 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-19T13:13:58.813 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-19T13:13:58.813 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-19T13:13:58.814 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-19T13:13:58.814 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-19T13:13:58.814 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-19T13:13:58.814 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.814 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.814 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-19T13:13:58.814 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-19T13:13:58.814 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-19T13:13:58.814 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.814 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-19T13:13:58.814 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-19T13:13:58.814 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-19T13:13:58.814 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-19T13:13:58.815 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-19T13:13:58.815 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-19T13:13:58.815 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-19T13:13:58.815 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-19T13:13:58.815 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-19T13:13:58.815 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-19T13:13:58.815 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-19T13:13:58.815 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-19T13:13:58.815 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-19T13:13:58.815 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-19T13:13:58.816 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.816 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.816 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:13:58.816 INFO:teuthology.orchestra.run.smithi103.stdout: re2-1:20211101-3.el9.x86_64 2024-08-19T13:13:58.816 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-19T13:13:58.816 INFO:teuthology.orchestra.run.smithi103.stdout: sqlite-3.34.1-7.el9.x86_64 2024-08-19T13:13:58.816 INFO:teuthology.orchestra.run.smithi103.stdout: sqlite-devel-3.34.1-7.el9.x86_64 2024-08-19T13:13:58.816 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-19T13:13:58.816 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-19T13:13:58.817 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:13:58.817 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-08-19T13:13:59.170 DEBUG:teuthology.parallel:result is None 2024-08-19T13:13:59.597 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 130/130 2024-08-19T13:13:59.597 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 1/130 2024-08-19T13:13:59.597 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2/130 2024-08-19T13:13:59.597 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 3/130 2024-08-19T13:13:59.597 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 4/130 2024-08-19T13:13:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 5/130 2024-08-19T13:13:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 6/130 2024-08-19T13:13:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 7/130 2024-08-19T13:13:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 8/130 2024-08-19T13:13:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 9/130 2024-08-19T13:13:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 10/130 2024-08-19T13:13:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 11/130 2024-08-19T13:13:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 12/130 2024-08-19T13:13:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 13/130 2024-08-19T13:13:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 14/130 2024-08-19T13:13:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 15/130 2024-08-19T13:13:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 16/130 2024-08-19T13:13:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 17/130 2024-08-19T13:13:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 18/130 2024-08-19T13:13:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 19/130 2024-08-19T13:13:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 20/130 2024-08-19T13:13:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 21/130 2024-08-19T13:13:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 22/130 2024-08-19T13:13:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 23/130 2024-08-19T13:13:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 24/130 2024-08-19T13:13:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 25/130 2024-08-19T13:13:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 26/130 2024-08-19T13:13:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 27/130 2024-08-19T13:13:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 28/130 2024-08-19T13:13:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 29/130 2024-08-19T13:13:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 30/130 2024-08-19T13:13:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 31/130 2024-08-19T13:13:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 32/130 2024-08-19T13:13:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 33/130 2024-08-19T13:13:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 34/130 2024-08-19T13:13:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 35/130 2024-08-19T13:13:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 36/130 2024-08-19T13:13:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 37/130 2024-08-19T13:13:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 38/130 2024-08-19T13:13:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 39/130 2024-08-19T13:13:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/130 2024-08-19T13:13:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/130 2024-08-19T13:13:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/130 2024-08-19T13:13:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/130 2024-08-19T13:13:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/130 2024-08-19T13:13:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 45/130 2024-08-19T13:13:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 46/130 2024-08-19T13:13:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 47/130 2024-08-19T13:13:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 48/130 2024-08-19T13:13:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 49/130 2024-08-19T13:13:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 50/130 2024-08-19T13:13:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 51/130 2024-08-19T13:13:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 52/130 2024-08-19T13:13:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/130 2024-08-19T13:13:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/130 2024-08-19T13:13:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 55/130 2024-08-19T13:13:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 56/130 2024-08-19T13:13:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 57/130 2024-08-19T13:13:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/130 2024-08-19T13:13:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-devel-3.9.19-4.el9.x86_64 59/130 2024-08-19T13:13:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 60/130 2024-08-19T13:13:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/130 2024-08-19T13:13:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/130 2024-08-19T13:13:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/130 2024-08-19T13:13:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/130 2024-08-19T13:13:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/130 2024-08-19T13:13:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/130 2024-08-19T13:13:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/130 2024-08-19T13:13:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/130 2024-08-19T13:13:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/130 2024-08-19T13:13:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/130 2024-08-19T13:13:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/130 2024-08-19T13:13:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/130 2024-08-19T13:13:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/130 2024-08-19T13:13:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/130 2024-08-19T13:13:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/130 2024-08-19T13:13:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/130 2024-08-19T13:13:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 77/130 2024-08-19T13:13:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 78/130 2024-08-19T13:13:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 79/130 2024-08-19T13:13:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 80/130 2024-08-19T13:13:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 81/130 2024-08-19T13:13:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : sqlite-devel-3.34.1-7.el9.x86_64 82/130 2024-08-19T13:13:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/130 2024-08-19T13:13:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 84/130 2024-08-19T13:13:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 85/130 2024-08-19T13:13:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 86/130 2024-08-19T13:13:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 87/130 2024-08-19T13:13:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 88/130 2024-08-19T13:13:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 89/130 2024-08-19T13:13:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 90/130 2024-08-19T13:13:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 91/130 2024-08-19T13:13:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 92/130 2024-08-19T13:13:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 93/130 2024-08-19T13:13:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 94/130 2024-08-19T13:13:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 95/130 2024-08-19T13:13:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 96/130 2024-08-19T13:13:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 97/130 2024-08-19T13:13:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 98/130 2024-08-19T13:13:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 99/130 2024-08-19T13:13:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 100/130 2024-08-19T13:13:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 101/130 2024-08-19T13:13:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 102/130 2024-08-19T13:13:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/130 2024-08-19T13:13:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 104/130 2024-08-19T13:13:59.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 105/130 2024-08-19T13:13:59.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 106/130 2024-08-19T13:13:59.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 107/130 2024-08-19T13:13:59.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 108/130 2024-08-19T13:13:59.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 109/130 2024-08-19T13:13:59.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 110/130 2024-08-19T13:13:59.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 111/130 2024-08-19T13:13:59.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 112/130 2024-08-19T13:13:59.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 113/130 2024-08-19T13:13:59.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 114/130 2024-08-19T13:13:59.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/130 2024-08-19T13:13:59.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 116/130 2024-08-19T13:13:59.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 117/130 2024-08-19T13:13:59.607 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 118/130 2024-08-19T13:13:59.607 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 119/130 2024-08-19T13:13:59.607 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 120/130 2024-08-19T13:13:59.607 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 121/130 2024-08-19T13:13:59.607 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 122/130 2024-08-19T13:13:59.607 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 123/130 2024-08-19T13:13:59.607 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 124/130 2024-08-19T13:13:59.607 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : re2-1:20211101-3.el9.x86_64 125/130 2024-08-19T13:13:59.607 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 126/130 2024-08-19T13:13:59.607 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 127/130 2024-08-19T13:13:59.608 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 128/130 2024-08-19T13:13:59.608 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 129/130 2024-08-19T13:14:01.486 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 130/130 2024-08-19T13:14:01.486 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:01.486 INFO:teuthology.orchestra.run.smithi122.stdout:Upgraded: 2024-08-19T13:14:01.486 INFO:teuthology.orchestra.run.smithi122.stdout: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.486 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.486 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-08-19T13:14:01.487 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-19T13:14:01.487 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-19T13:14:01.487 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.487 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.487 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.487 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.487 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:14:01.487 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.487 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.487 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.487 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:14:01.487 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:14:01.488 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:14:01.488 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:14:01.488 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:14:01.488 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.488 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.488 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:14:01.488 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.488 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.488 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.488 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:14:01.488 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T13:14:01.488 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-19T13:14:01.488 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-19T13:14:01.489 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-19T13:14:01.489 INFO:teuthology.orchestra.run.smithi122.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-19T13:14:01.489 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-19T13:14:01.489 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-19T13:14:01.489 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-19T13:14:01.489 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-19T13:14:01.489 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.489 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.489 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.489 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-19T13:14:01.489 INFO:teuthology.orchestra.run.smithi122.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-19T13:14:01.489 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-19T13:14:01.490 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-19T13:14:01.490 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-19T13:14:01.490 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.490 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.490 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-19T13:14:01.490 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.490 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-19T13:14:01.490 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-19T13:14:01.490 INFO:teuthology.orchestra.run.smithi122.stdout: lua-5.4.4-4.el9.x86_64 2024-08-19T13:14:01.490 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-19T13:14:01.490 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-19T13:14:01.490 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-19T13:14:01.490 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-19T13:14:01.491 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-19T13:14:01.491 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-19T13:14:01.491 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-19T13:14:01.491 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-19T13:14:01.491 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-19T13:14:01.491 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-19T13:14:01.491 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-19T13:14:01.491 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.491 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.491 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.491 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-19T13:14:01.491 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-19T13:14:01.492 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-19T13:14:01.492 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-19T13:14:01.492 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-19T13:14:01.492 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel-3.9.19-4.el9.x86_64 2024-08-19T13:14:01.492 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-19T13:14:01.492 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-19T13:14:01.492 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-19T13:14:01.492 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-19T13:14:01.492 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-19T13:14:01.492 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-19T13:14:01.492 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-19T13:14:01.492 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-19T13:14:01.492 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-19T13:14:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-19T13:14:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-19T13:14:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-19T13:14:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-19T13:14:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-19T13:14:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-19T13:14:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-19T13:14:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-19T13:14:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-19T13:14:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-19T13:14:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-19T13:14:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-19T13:14:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-19T13:14:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply-3.11-14.el9.noarch 2024-08-19T13:14:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-19T13:14:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-19T13:14:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-19T13:14:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-19T13:14:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-19T13:14:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-19T13:14:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-19T13:14:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-19T13:14:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-19T13:14:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-19T13:14:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-19T13:14:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-19T13:14:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-19T13:14:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-19T13:14:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-19T13:14:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-19T13:14:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-19T13:14:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-19T13:14:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-19T13:14:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-19T13:14:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-19T13:14:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-19T13:14:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-19T13:14:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-19T13:14:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-19T13:14:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T13:14:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: re2-1:20211101-3.el9.x86_64 2024-08-19T13:14:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-19T13:14:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: sqlite-3.34.1-7.el9.x86_64 2024-08-19T13:14:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: sqlite-devel-3.34.1-7.el9.x86_64 2024-08-19T13:14:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-19T13:14:01.497 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-19T13:14:01.497 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:01.497 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-08-19T13:14:01.870 DEBUG:teuthology.parallel:result is None 2024-08-19T13:14:01.870 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T13:14:02.021 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-19T13:14:02.048 INFO:teuthology.orchestra.run.smithi103.stdout:19.1.0-1382.gde09dd01.el9 2024-08-19T13:14:02.048 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1382.gde09dd01.el9 2024-08-19T13:14:02.049 INFO:teuthology.task.install:The correct ceph version 19.1.0-1382.gde09dd01 is installed. 2024-08-19T13:14:02.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T13:14:02.197 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-19T13:14:02.220 INFO:teuthology.orchestra.run.smithi122.stdout:19.1.0-1382.gde09dd01.el9 2024-08-19T13:14:02.220 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1382.gde09dd01.el9 2024-08-19T13:14:02.220 INFO:teuthology.task.install:The correct ceph version 19.1.0-1382.gde09dd01 is installed. 2024-08-19T13:14:02.222 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-19T13:14:02.222 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:02.222 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-19T13:14:02.255 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:02.255 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-19T13:14:02.287 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-19T13:14:02.288 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:02.288 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-08-19T13:14:02.323 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-19T13:14:02.390 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:02.390 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-08-19T13:14:02.415 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-19T13:14:02.481 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-19T13:14:02.481 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:02.481 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-19T13:14:02.508 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-19T13:14:02.576 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:02.576 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-19T13:14:02.600 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-19T13:14:02.665 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-19T13:14:02.665 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:02.665 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-08-19T13:14:02.692 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-19T13:14:02.758 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:02.758 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-08-19T13:14:02.783 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-19T13:14:02.848 INFO:teuthology.run_tasks:Running task ceph... 2024-08-19T13:14:02.934 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-08-19T13:14:02.934 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /var/log/ceph 2024-08-19T13:14:02.937 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /var/log/ceph 2024-08-19T13:14:02.961 INFO:tasks.ceph:Disabling ceph logrotate... 2024-08-19T13:14:02.961 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-19T13:14:03.003 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-19T13:14:03.027 INFO:tasks.ceph:Creating extra log directories... 2024-08-19T13:14:03.027 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-19T13:14:03.068 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-19T13:14:03.103 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-08-19T13:14:03.103 INFO:tasks.ceph:config {'conf': {'client': {'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-08-19T13:14:03.103 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi/7859586', 'branch': 'squid', 'description': 'rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} tasks/libcephsqlite}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7859586', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled'], 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '78215f5a1cf5eaf321d506a66539942e12851f34'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 700, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 3241, 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'sleep_before_teardown': 0, 'subset': '18710/100000', 'suite': 'rados', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '78215f5a1cf5eaf321d506a66539942e12851f34', 'targets': {'smithi103.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIRjIVoA1i5YJhna8z+/nKH6ZE4K6G3Em9WsOE+j6N6m+VaSRIpQ6oLuTmC//A7MaBv6h0IbMSUWknNbWv/KT38=', 'smithi122.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNJpH5obQCcY3ccTGPYbIDaDbAgIFhw+7EvrnGx1UzW3YvudX13ku8x9Y2fGckrhgbSMmYQDFybwaM9IeDWFeAA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833'}}, {'ceph': None}, {'exec': {'client.0': ['ceph osd pool create cephsqlite', "ceph auth get-or-create client.libcephsqlite mon 'profile simple-rados-client-with-blocklist' osd 'allow rwx pool=cephsqlite' >> /etc/ceph/ceph.keyring"]}}, {'exec': {'client.0': ['ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr']}}, {'workunit': {'clients': {'client.0': ['rados/test_libcephsqlite.sh cephsqlite']}, 'env': {'CEPH_ARGS': '--id libcephsqlite --no-log-to-stderr'}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '67db276914d29505c9707d04f3a3b8fb9d8aee4b', 'timestamp': '2024-08-18_21:00:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-08-19T13:14:03.104 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-19T13:14:03.135 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-19T13:14:03.150 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m0777 -- /var/run/ceph 2024-08-19T13:14:03.191 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2024-08-19T13:14:03.215 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:03.215 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-08-19T13:14:03.269 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-19T13:14:03.269 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-08-19T13:14:03.325 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-19T13:14:03.325 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T13:14:03.325 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-08-19T13:14:03.325 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T13:14:03.325 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T13:14:03.325 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-08-19 13:13:52.063414001 +0000 2024-08-19T13:14:03.325 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-08-19 13:12:15.307417545 +0000 2024-08-19T13:14:03.325 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-08-19 13:12:15.307417545 +0000 2024-08-19T13:14:03.325 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-08-19 13:12:15.307417545 +0000 2024-08-19T13:14:03.326 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-19T13:14:03.388 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-08-19T13:14:03.389 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-08-19T13:14:03.389 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000202167 s, 2.5 MB/s 2024-08-19T13:14:03.390 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-19T13:14:03.447 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-08-19T13:14:03.502 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-19T13:14:03.502 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T13:14:03.502 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-08-19T13:14:03.503 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T13:14:03.503 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T13:14:03.503 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-08-19 13:13:52.063414001 +0000 2024-08-19T13:14:03.503 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-08-19 13:12:15.654414693 +0000 2024-08-19T13:14:03.503 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-08-19 13:12:15.654414693 +0000 2024-08-19T13:14:03.503 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-08-19 13:12:15.654414693 +0000 2024-08-19T13:14:03.503 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-19T13:14:03.565 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-08-19T13:14:03.566 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-08-19T13:14:03.566 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.00022396 s, 2.3 MB/s 2024-08-19T13:14:03.567 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-19T13:14:03.623 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-08-19T13:14:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-19T13:14:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T13:14:03.679 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-08-19T13:14:03.679 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T13:14:03.679 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T13:14:03.679 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-08-19 13:13:52.063414001 +0000 2024-08-19T13:14:03.679 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-08-19 13:12:15.969412104 +0000 2024-08-19T13:14:03.679 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-08-19 13:12:15.969412104 +0000 2024-08-19T13:14:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-08-19 13:12:15.969412104 +0000 2024-08-19T13:14:03.680 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-19T13:14:03.742 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-08-19T13:14:03.742 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-08-19T13:14:03.742 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000166658 s, 3.1 MB/s 2024-08-19T13:14:03.743 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-19T13:14:03.800 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-08-19T13:14:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-19T13:14:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T13:14:03.856 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-08-19T13:14:03.856 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T13:14:03.856 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T13:14:03.856 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-08-19 13:13:52.064413981 +0000 2024-08-19T13:14:03.856 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-08-19 13:12:16.316409252 +0000 2024-08-19T13:14:03.856 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-08-19 13:12:16.316409252 +0000 2024-08-19T13:14:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-08-19 13:12:16.316409252 +0000 2024-08-19T13:14:03.857 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-19T13:14:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-08-19T13:14:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-08-19T13:14:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000162838 s, 3.1 MB/s 2024-08-19T13:14:03.920 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-19T13:14:03.977 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-19T13:14:03.977 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:03.977 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-08-19T13:14:03.993 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-19T13:14:03.993 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-08-19T13:14:04.049 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-19T13:14:04.049 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T13:14:04.049 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-08-19T13:14:04.049 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T13:14:04.049 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T13:14:04.049 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-08-19 13:13:54.494656336 +0000 2024-08-19T13:14:04.049 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-08-19 13:12:15.319010609 +0000 2024-08-19T13:14:04.049 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-08-19 13:12:15.319010609 +0000 2024-08-19T13:14:04.049 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-08-19 13:12:15.319010609 +0000 2024-08-19T13:14:04.049 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-19T13:14:04.112 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-08-19T13:14:04.112 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-08-19T13:14:04.112 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000260684 s, 2.0 MB/s 2024-08-19T13:14:04.113 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-19T13:14:04.170 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-08-19T13:14:04.225 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-19T13:14:04.226 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T13:14:04.226 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-08-19T13:14:04.226 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T13:14:04.226 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T13:14:04.226 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-08-19 13:13:54.494656336 +0000 2024-08-19T13:14:04.226 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-08-19 13:12:15.654004498 +0000 2024-08-19T13:14:04.226 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-08-19 13:12:15.654004498 +0000 2024-08-19T13:14:04.226 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-08-19 13:12:15.654004498 +0000 2024-08-19T13:14:04.226 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-19T13:14:04.289 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-08-19T13:14:04.289 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-08-19T13:14:04.289 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000195012 s, 2.6 MB/s 2024-08-19T13:14:04.290 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-19T13:14:04.347 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-08-19T13:14:04.402 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-19T13:14:04.402 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T13:14:04.402 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-08-19T13:14:04.402 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T13:14:04.402 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T13:14:04.402 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-08-19 13:13:54.494656336 +0000 2024-08-19T13:14:04.402 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-08-19 13:12:15.988998387 +0000 2024-08-19T13:14:04.402 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-08-19 13:12:15.988998387 +0000 2024-08-19T13:14:04.402 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-08-19 13:12:15.988998387 +0000 2024-08-19T13:14:04.403 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-19T13:14:04.465 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-08-19T13:14:04.465 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-08-19T13:14:04.465 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000211434 s, 2.4 MB/s 2024-08-19T13:14:04.466 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-19T13:14:04.523 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-08-19T13:14:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-19T13:14:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T13:14:04.579 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-08-19T13:14:04.579 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T13:14:04.579 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T13:14:04.579 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-08-19 13:13:54.495656313 +0000 2024-08-19T13:14:04.579 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-08-19 13:12:16.307992568 +0000 2024-08-19T13:14:04.580 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-08-19 13:12:16.307992568 +0000 2024-08-19T13:14:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-08-19 13:12:16.307992568 +0000 2024-08-19T13:14:04.580 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-19T13:14:04.642 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-08-19T13:14:04.643 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-08-19T13:14:04.643 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000163 s, 3.1 MB/s 2024-08-19T13:14:04.644 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-19T13:14:04.699 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-19T13:14:04.700 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi103.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-08-19T13:14:04.700 INFO:tasks.ceph:Generating config... 2024-08-19T13:14:04.701 INFO:tasks.ceph:[client] debug cephsqlite = 20 2024-08-19T13:14:04.701 INFO:tasks.ceph:[client] debug client = 20 2024-08-19T13:14:04.701 INFO:tasks.ceph:[client] debug ms = 1 2024-08-19T13:14:04.701 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-08-19T13:14:04.701 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-08-19T13:14:04.701 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-08-19T13:14:04.701 INFO:tasks.ceph:[global] ms type = async 2024-08-19T13:14:04.701 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-08-19T13:14:04.701 INFO:tasks.ceph:[mgr] debug ms = 1 2024-08-19T13:14:04.702 INFO:tasks.ceph:[mon] debug mon = 20 2024-08-19T13:14:04.702 INFO:tasks.ceph:[mon] debug ms = 1 2024-08-19T13:14:04.702 INFO:tasks.ceph:[mon] debug paxos = 20 2024-08-19T13:14:04.702 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-08-19T13:14:04.702 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-08-19T13:14:04.702 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-08-19T13:14:04.702 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-08-19T13:14:04.702 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-08-19T13:14:04.702 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-08-19T13:14:04.702 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-08-19T13:14:04.702 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2024-08-19T13:14:04.702 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-08-19T13:14:04.703 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-08-19T13:14:04.703 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-08-19T13:14:04.703 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-08-19T13:14:04.704 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-08-19T13:14:04.705 INFO:tasks.ceph:[osd] debug ms = 1 2024-08-19T13:14:04.705 INFO:tasks.ceph:[osd] debug osd = 20 2024-08-19T13:14:04.705 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-08-19T13:14:04.705 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-08-19T13:14:04.705 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-08-19T13:14:04.705 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-08-19T13:14:04.705 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-08-19T13:14:04.705 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-08-19T13:14:04.705 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-08-19T13:14:04.705 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-08-19T13:14:04.705 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-08-19T13:14:04.705 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-08-19T13:14:04.705 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-08-19T13:14:04.706 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-08-19T13:14:04.706 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-08-19T13:14:04.706 INFO:tasks.ceph:Setting up mon.a... 2024-08-19T13:14:04.706 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-08-19T13:14:04.780 INFO:teuthology.orchestra.run.smithi103.stdout:creating /etc/ceph/ceph.keyring 2024-08-19T13:14:04.783 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-08-19T13:14:04.829 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-19T13:14:04.854 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.103'), ('mon.c', '[v2:172.21.15.103:3301,v1:172.21.15.103:6790]'), ('mon.b', '172.21.15.122')] 2024-08-19T13:14:04.855 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.103,[v2:172.21.15.103:3301,v1:172.21.15.103:6790],172.21.15.122', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-08-19T13:14:04.856 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:04.856 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-08-19T13:14:04.910 DEBUG:teuthology.orchestra.run.smithi103:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.103 --addv c '[v2:172.21.15.103:3301,v1:172.21.15.103:6790]' --add b 172.21.15.122 --print /home/ubuntu/cephtest/ceph.monmap 2024-08-19T13:14:04.997 INFO:teuthology.orchestra.run.smithi103.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-08-19T13:14:04.998 INFO:teuthology.orchestra.run.smithi103.stdout:monmaptool: generated fsid 9a6e456b-d04b-48cb-846f-0fbd1ee79f13 2024-08-19T13:14:04.998 INFO:teuthology.orchestra.run.smithi103.stdout:setting min_mon_release = quincy 2024-08-19T13:14:04.998 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 0 2024-08-19T13:14:04.998 INFO:teuthology.orchestra.run.smithi103.stdout:fsid 9a6e456b-d04b-48cb-846f-0fbd1ee79f13 2024-08-19T13:14:04.998 INFO:teuthology.orchestra.run.smithi103.stdout:last_changed 2024-08-19T13:14:04.997914+0000 2024-08-19T13:14:04.998 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-08-19T13:14:04.997914+0000 2024-08-19T13:14:04.998 INFO:teuthology.orchestra.run.smithi103.stdout:min_mon_release 17 (quincy) 2024-08-19T13:14:04.998 INFO:teuthology.orchestra.run.smithi103.stdout:election_strategy: 3 2024-08-19T13:14:04.998 INFO:teuthology.orchestra.run.smithi103.stdout:0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-08-19T13:14:04.998 INFO:teuthology.orchestra.run.smithi103.stdout:1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.b 2024-08-19T13:14:04.998 INFO:teuthology.orchestra.run.smithi103.stdout:2: [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] mon.c 2024-08-19T13:14:04.999 INFO:teuthology.orchestra.run.smithi103.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-08-19T13:14:05.000 DEBUG:teuthology.orchestra.run.smithi103:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-08-19T13:14:05.015 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9a6e456b-d04b-48cb-846f-0fbd1ee79f13... 2024-08-19T13:14:05.016 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-19T13:14:05.058 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-19T13:14:05.098 INFO:teuthology.orchestra.run.smithi103.stdout:[global] 2024-08-19T13:14:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: chdir = "" 2024-08-19T13:14:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-19T13:14:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: auth supported = cephx 2024-08-19T13:14:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: filestore xattr use omap = true 2024-08-19T13:14:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: mon clock drift allowed = 1.000 2024-08-19T13:14:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: osd crush chooseleaf type = 0 2024-08-19T13:14:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: auth debug = true 2024-08-19T13:14:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: ms die on old message = true 2024-08-19T13:14:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: ms die on bug = true 2024-08-19T13:14:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-19T13:14:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: mon pg warn max object skew = 0 2024-08-19T13:14:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: # disable pg_autoscaler by default for new pools 2024-08-19T13:14:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-19T13:14:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: osd pool default size = 2 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd allow primary affinity = true 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd allow pg remap = true 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on legacy crush tunables = false 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on crush straw calc version zero = false 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on no sortbitwise = false 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on osd down out interval zero = false 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on too few osds = false 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: mon_allow_pool_size_one = true 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: osd default data pool replay window = 5 2024-08-19T13:14:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: mon allow pool delete = true 2024-08-19T13:14:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: mon cluster log file level = debug 2024-08-19T13:14:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: debug asserts on shutdown = true 2024-08-19T13:14:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: mon health detail to clog = false 2024-08-19T13:14:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: mon host = "172.21.15.103,[v2:172.21.15.103:3301,v1:172.21.15.103:6790],172.21.15.122" 2024-08-19T13:14:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: mon client directed command retry = 5 2024-08-19T13:14:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: mon election default strategy = 3 2024-08-19T13:14:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: ms inject socket failures = 5000 2024-08-19T13:14:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: ms type = async 2024-08-19T13:14:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: fsid = 9a6e456b-d04b-48cb-846f-0fbd1ee79f13 2024-08-19T13:14:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.101 INFO:teuthology.orchestra.run.smithi103.stdout:[osd] 2024-08-19T13:14:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: osd journal size = 100 2024-08-19T13:14:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: osd scrub load threshold = 5.0 2024-08-19T13:14:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: osd scrub max interval = 600 2024-08-19T13:14:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: osd mclock profile = high_recovery_ops 2024-08-19T13:14:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: osd recover clone overlap = true 2024-08-19T13:14:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: osd recovery max chunk = 1048576 2024-08-19T13:14:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug shutdown = true 2024-08-19T13:14:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug op order = true 2024-08-19T13:14:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug verify stray on activate = true 2024-08-19T13:14:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug trim objects = true 2024-08-19T13:14:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: osd open classes on start = true 2024-08-19T13:14:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug pg log writeout = true 2024-08-19T13:14:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: osd deep scrub update digest min age = 30 2024-08-19T13:14:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: osd map max advance = 10 2024-08-19T13:14:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: journal zero on create = true 2024-08-19T13:14:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: filestore ondisk finisher threads = 3 2024-08-19T13:14:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: filestore apply finisher threads = 3 2024-08-19T13:14:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: bdev debug aio = true 2024-08-19T13:14:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug misdirected ops = true 2024-08-19T13:14:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore block size = 96636764160 2024-08-19T13:14:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore compression algorithm = snappy 2024-08-19T13:14:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore compression mode = aggressive 2024-08-19T13:14:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore fsck on mount = True 2024-08-19T13:14:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore zero block detection = True 2024-08-19T13:14:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: debug bluefs = 20 2024-08-19T13:14:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: debug bluestore = 20 2024-08-19T13:14:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2024-08-19T13:14:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: debug osd = 20 2024-08-19T13:14:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: debug rocksdb = 10 2024-08-19T13:14:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd backfillfull_ratio = 0.85 2024-08-19T13:14:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd full ratio = 0.9 2024-08-19T13:14:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd nearfull ratio = 0.8 2024-08-19T13:14:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug verify cached snaps = True 2024-08-19T13:14:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug verify missing on start = True 2024-08-19T13:14:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: osd failsafe full ratio = 0.95 2024-08-19T13:14:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: osd mclock override recovery settings = True 2024-08-19T13:14:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: osd objectstore = bluestore 2024-08-19T13:14:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: osd op queue = debug_random 2024-08-19T13:14:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: osd op queue cut off = debug_random 2024-08-19T13:14:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: osd shutdown pgref assert = True 2024-08-19T13:14:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.106 INFO:teuthology.orchestra.run.smithi103.stdout:[mgr] 2024-08-19T13:14:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2024-08-19T13:14:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: debug mgr = 20 2024-08-19T13:14:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: debug mon = 20 2024-08-19T13:14:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: debug auth = 20 2024-08-19T13:14:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min pgs per osd = 4 2024-08-19T13:14:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min bytes per osd = 10 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout: mgr/telemetry/nag = false 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout:[mon] 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout: debug mon = 20 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout: debug paxos = 20 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout: debug auth = 20 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout: mon data avail warn = 5 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout: mon mgr mkfs grace = 240 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min pgs per osd = 4 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd reporter subtree level = osd 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd prime pg temp = true 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min bytes per osd = 10 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.107 INFO:teuthology.orchestra.run.smithi103.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: auth mon ticket ttl = 660 # 11m 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: auth service ticket ttl = 240 # 4m 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: # don't complain about insecure global_id in the test suite 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: # 1m isn't quite enough 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: mon_down_mkfs_grace = 2m 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_filestore_osds = false 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: mon min osdmap epochs = 50 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: mon osdmap full prune interval = 2 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: mon osdmap full prune min = 15 2024-08-19T13:14:05.108 INFO:teuthology.orchestra.run.smithi103.stdout: mon osdmap full prune txsize = 2 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout: mon scrub interval = 300 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout: paxos service trim min = 10 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout:[client] 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout: rgw cache enabled = true 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout: rgw enable ops log = true 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout: rgw enable usage log = true 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout: debug cephsqlite = 20 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout: debug client = 20 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout:[mon.a] 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout:[mon.c] 2024-08-19T13:14:05.109 INFO:teuthology.orchestra.run.smithi103.stdout:[mon.b] 2024-08-19T13:14:05.110 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-08-19T13:14:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: chdir = "" 2024-08-19T13:14:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-19T13:14:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: auth supported = cephx 2024-08-19T13:14:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: filestore xattr use omap = true 2024-08-19T13:14:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: mon clock drift allowed = 1.000 2024-08-19T13:14:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: osd crush chooseleaf type = 0 2024-08-19T13:14:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: auth debug = true 2024-08-19T13:14:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on old message = true 2024-08-19T13:14:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on bug = true 2024-08-19T13:14:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-19T13:14:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: mon pg warn max object skew = 0 2024-08-19T13:14:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: # disable pg_autoscaler by default for new pools 2024-08-19T13:14:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-19T13:14:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default size = 2 2024-08-19T13:14:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow primary affinity = true 2024-08-19T13:14:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow pg remap = true 2024-08-19T13:14:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on legacy crush tunables = false 2024-08-19T13:14:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on crush straw calc version zero = false 2024-08-19T13:14:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on no sortbitwise = false 2024-08-19T13:14:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on osd down out interval zero = false 2024-08-19T13:14:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on too few osds = false 2024-08-19T13:14:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-19T13:14:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-19T13:14:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: mon_allow_pool_size_one = true 2024-08-19T13:14:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-19T13:14:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: osd default data pool replay window = 5 2024-08-19T13:14:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: mon allow pool delete = true 2024-08-19T13:14:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: mon cluster log file level = debug 2024-08-19T13:14:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: debug asserts on shutdown = true 2024-08-19T13:14:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: mon health detail to clog = false 2024-08-19T13:14:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: mon host = "172.21.15.103,[v2:172.21.15.103:3301,v1:172.21.15.103:6790],172.21.15.122" 2024-08-19T13:14:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: mon client directed command retry = 5 2024-08-19T13:14:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: mon election default strategy = 3 2024-08-19T13:14:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: ms inject socket failures = 5000 2024-08-19T13:14:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: ms type = async 2024-08-19T13:14:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = 9a6e456b-d04b-48cb-846f-0fbd1ee79f13 2024-08-19T13:14:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.116 INFO:teuthology.orchestra.run.smithi122.stdout:[osd] 2024-08-19T13:14:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: osd journal size = 100 2024-08-19T13:14:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub load threshold = 5.0 2024-08-19T13:14:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub max interval = 600 2024-08-19T13:14:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock profile = high_recovery_ops 2024-08-19T13:14:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: osd recover clone overlap = true 2024-08-19T13:14:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: osd recovery max chunk = 1048576 2024-08-19T13:14:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug shutdown = true 2024-08-19T13:14:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug op order = true 2024-08-19T13:14:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify stray on activate = true 2024-08-19T13:14:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug trim objects = true 2024-08-19T13:14:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: osd open classes on start = true 2024-08-19T13:14:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug pg log writeout = true 2024-08-19T13:14:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: osd deep scrub update digest min age = 30 2024-08-19T13:14:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: osd map max advance = 10 2024-08-19T13:14:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: journal zero on create = true 2024-08-19T13:14:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: filestore ondisk finisher threads = 3 2024-08-19T13:14:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: filestore apply finisher threads = 3 2024-08-19T13:14:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: bdev debug aio = true 2024-08-19T13:14:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug misdirected ops = true 2024-08-19T13:14:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore block size = 96636764160 2024-08-19T13:14:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore compression algorithm = snappy 2024-08-19T13:14:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore compression mode = aggressive 2024-08-19T13:14:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore fsck on mount = True 2024-08-19T13:14:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore zero block detection = True 2024-08-19T13:14:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluefs = 20 2024-08-19T13:14:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluestore = 20 2024-08-19T13:14:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-08-19T13:14:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: debug osd = 20 2024-08-19T13:14:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: debug rocksdb = 10 2024-08-19T13:14:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd backfillfull_ratio = 0.85 2024-08-19T13:14:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd full ratio = 0.9 2024-08-19T13:14:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd nearfull ratio = 0.8 2024-08-19T13:14:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify cached snaps = True 2024-08-19T13:14:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify missing on start = True 2024-08-19T13:14:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: osd failsafe full ratio = 0.95 2024-08-19T13:14:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock override recovery settings = True 2024-08-19T13:14:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: osd objectstore = bluestore 2024-08-19T13:14:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: osd op queue = debug_random 2024-08-19T13:14:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: osd op queue cut off = debug_random 2024-08-19T13:14:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: osd shutdown pgref assert = True 2024-08-19T13:14:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.122 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr] 2024-08-19T13:14:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-08-19T13:14:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: debug mgr = 20 2024-08-19T13:14:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-08-19T13:14:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-08-19T13:14:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-08-19T13:14:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-08-19T13:14:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: mgr/telemetry/nag = false 2024-08-19T13:14:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.123 INFO:teuthology.orchestra.run.smithi122.stdout:[mon] 2024-08-19T13:14:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-08-19T13:14:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-08-19T13:14:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: debug paxos = 20 2024-08-19T13:14:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-08-19T13:14:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: mon data avail warn = 5 2024-08-19T13:14:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: mon mgr mkfs grace = 240 2024-08-19T13:14:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-08-19T13:14:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd reporter subtree level = osd 2024-08-19T13:14:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd prime pg temp = true 2024-08-19T13:14:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-08-19T13:14:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-19T13:14:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: auth mon ticket ttl = 660 # 11m 2024-08-19T13:14:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: auth service ticket ttl = 240 # 4m 2024-08-19T13:14:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: # don't complain about insecure global_id in the test suite 2024-08-19T13:14:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-19T13:14:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-19T13:14:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: # 1m isn't quite enough 2024-08-19T13:14:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: mon_down_mkfs_grace = 2m 2024-08-19T13:14:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_filestore_osds = false 2024-08-19T13:14:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: mon min osdmap epochs = 50 2024-08-19T13:14:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: mon osdmap full prune interval = 2 2024-08-19T13:14:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: mon osdmap full prune min = 15 2024-08-19T13:14:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: mon osdmap full prune txsize = 2 2024-08-19T13:14:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: mon scrub interval = 300 2024-08-19T13:14:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: paxos service trim min = 10 2024-08-19T13:14:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-08-19T13:14:05.126 INFO:teuthology.orchestra.run.smithi122.stdout:[client] 2024-08-19T13:14:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: rgw cache enabled = true 2024-08-19T13:14:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable ops log = true 2024-08-19T13:14:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable usage log = true 2024-08-19T13:14:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-19T13:14:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-19T13:14:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: debug cephsqlite = 20 2024-08-19T13:14:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: debug client = 20 2024-08-19T13:14:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-08-19T13:14:05.127 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.a] 2024-08-19T13:14:05.127 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.c] 2024-08-19T13:14:05.127 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.b] 2024-08-19T13:14:05.128 INFO:tasks.ceph:Creating admin key on mon.a... 2024-08-19T13:14:05.129 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-08-19T13:14:05.223 INFO:tasks.ceph:Copying monmap to all nodes... 2024-08-19T13:14:05.223 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:05.223 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-08-19T13:14:05.239 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:05.240 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-08-19T13:14:05.295 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T13:14:05.295 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:05.295 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-19T13:14:05.295 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-19T13:14:05.370 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:05.370 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-19T13:14:05.426 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi122.front.sepia.ceph.com 2024-08-19T13:14:05.427 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:05.427 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-19T13:14:05.427 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-19T13:14:05.462 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:05.462 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-19T13:14:05.518 INFO:tasks.ceph:Setting up mon nodes... 2024-08-19T13:14:05.518 INFO:tasks.ceph:Setting up mgr nodes... 2024-08-19T13:14:05.518 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-08-19T13:14:05.570 INFO:teuthology.orchestra.run.smithi103.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-08-19T13:14:05.573 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-08-19T13:14:05.636 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-08-19T13:14:05.639 INFO:tasks.ceph:Setting up mds nodes... 2024-08-19T13:14:05.639 INFO:tasks.ceph_client:Setting up client nodes... 2024-08-19T13:14:05.639 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-19T13:14:05.680 INFO:teuthology.orchestra.run.smithi103.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-08-19T13:14:05.693 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-08-19T13:14:05.734 INFO:teuthology.orchestra.run.smithi122.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-08-19T13:14:05.746 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-08-19T13:14:05.746 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi103.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-08-19T13:14:05.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-08-19T13:14:05.771 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-19T13:14:05.771 INFO:tasks.ceph:role: osd.0 2024-08-19T13:14:05.771 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T13:14:05.771 DEBUG:teuthology.orchestra.run.smithi103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-19T13:14:05.843 INFO:teuthology.orchestra.run.smithi103.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-19T13:14:05.843 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-19T13:14:05.843 INFO:teuthology.orchestra.run.smithi103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-19T13:14:05.843 INFO:teuthology.orchestra.run.smithi103.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-19T13:14:05.843 INFO:teuthology.orchestra.run.smithi103.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-19T13:14:05.843 INFO:teuthology.orchestra.run.smithi103.stdout: = sunit=0 swidth=0 blks 2024-08-19T13:14:05.844 INFO:teuthology.orchestra.run.smithi103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-19T13:14:05.844 INFO:teuthology.orchestra.run.smithi103.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-19T13:14:05.844 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-19T13:14:05.844 INFO:teuthology.orchestra.run.smithi103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-19T13:14:06.354 INFO:teuthology.orchestra.run.smithi103.stdout:Discarding blocks...Done. 2024-08-19T13:14:06.355 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi103.front.sepia.ceph.com -o noatime 2024-08-19T13:14:06.356 DEBUG:teuthology.orchestra.run.smithi103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-08-19T13:14:06.490 DEBUG:teuthology.orchestra.run.smithi103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-08-19T13:14:06.525 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-08-19T13:14:06.591 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-19T13:14:06.591 INFO:tasks.ceph:role: osd.1 2024-08-19T13:14:06.591 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T13:14:06.591 DEBUG:teuthology.orchestra.run.smithi103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-19T13:14:06.663 INFO:teuthology.orchestra.run.smithi103.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-19T13:14:06.664 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-19T13:14:06.664 INFO:teuthology.orchestra.run.smithi103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-19T13:14:06.664 INFO:teuthology.orchestra.run.smithi103.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-19T13:14:06.664 INFO:teuthology.orchestra.run.smithi103.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-19T13:14:06.664 INFO:teuthology.orchestra.run.smithi103.stdout: = sunit=0 swidth=0 blks 2024-08-19T13:14:06.664 INFO:teuthology.orchestra.run.smithi103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-19T13:14:06.664 INFO:teuthology.orchestra.run.smithi103.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-19T13:14:06.664 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-19T13:14:06.664 INFO:teuthology.orchestra.run.smithi103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-19T13:14:07.112 INFO:teuthology.orchestra.run.smithi103.stdout:Discarding blocks...Done. 2024-08-19T13:14:07.114 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi103.front.sepia.ceph.com -o noatime 2024-08-19T13:14:07.114 DEBUG:teuthology.orchestra.run.smithi103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-08-19T13:14:07.207 DEBUG:teuthology.orchestra.run.smithi103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-08-19T13:14:07.236 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-08-19T13:14:07.302 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-19T13:14:07.302 INFO:tasks.ceph:role: osd.2 2024-08-19T13:14:07.302 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T13:14:07.302 DEBUG:teuthology.orchestra.run.smithi103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-19T13:14:07.375 INFO:teuthology.orchestra.run.smithi103.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-19T13:14:07.375 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-19T13:14:07.375 INFO:teuthology.orchestra.run.smithi103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-19T13:14:07.375 INFO:teuthology.orchestra.run.smithi103.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-19T13:14:07.375 INFO:teuthology.orchestra.run.smithi103.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-19T13:14:07.375 INFO:teuthology.orchestra.run.smithi103.stdout: = sunit=0 swidth=0 blks 2024-08-19T13:14:07.375 INFO:teuthology.orchestra.run.smithi103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-19T13:14:07.375 INFO:teuthology.orchestra.run.smithi103.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-19T13:14:07.376 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-19T13:14:07.376 INFO:teuthology.orchestra.run.smithi103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-19T13:14:07.855 INFO:teuthology.orchestra.run.smithi103.stdout:Discarding blocks...Done. 2024-08-19T13:14:07.857 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi103.front.sepia.ceph.com -o noatime 2024-08-19T13:14:07.857 DEBUG:teuthology.orchestra.run.smithi103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-08-19T13:14:07.976 DEBUG:teuthology.orchestra.run.smithi103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-08-19T13:14:08.008 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-08-19T13:14:08.074 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-19T13:14:08.074 INFO:tasks.ceph:role: osd.3 2024-08-19T13:14:08.075 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T13:14:08.075 DEBUG:teuthology.orchestra.run.smithi103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-08-19T13:14:08.146 INFO:teuthology.orchestra.run.smithi103.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-08-19T13:14:08.146 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-19T13:14:08.146 INFO:teuthology.orchestra.run.smithi103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-19T13:14:08.147 INFO:teuthology.orchestra.run.smithi103.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-19T13:14:08.147 INFO:teuthology.orchestra.run.smithi103.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-19T13:14:08.147 INFO:teuthology.orchestra.run.smithi103.stdout: = sunit=0 swidth=0 blks 2024-08-19T13:14:08.147 INFO:teuthology.orchestra.run.smithi103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-19T13:14:08.147 INFO:teuthology.orchestra.run.smithi103.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-19T13:14:08.147 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-19T13:14:08.147 INFO:teuthology.orchestra.run.smithi103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-19T13:14:08.679 INFO:teuthology.orchestra.run.smithi103.stdout:Discarding blocks...Done. 2024-08-19T13:14:08.681 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi103.front.sepia.ceph.com -o noatime 2024-08-19T13:14:08.681 DEBUG:teuthology.orchestra.run.smithi103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-08-19T13:14:08.790 DEBUG:teuthology.orchestra.run.smithi103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-08-19T13:14:08.821 DEBUG:teuthology.orchestra.run.smithi103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-19T13:14:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:08.919+0000 7f8c2f21b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-08-19T13:14:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:08.919+0000 7f8c2f21b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-08-19T13:14:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:08.919+0000 7f8c2f21b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-19T13:14:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:08.919+0000 7f8c2f21b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-19T13:14:08.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:08.919+0000 7f8c2f21b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-19T13:14:08.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:08.920+0000 7f8c2f21b640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-08-19T13:14:11.777 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-08-19T13:14:11.809 DEBUG:teuthology.orchestra.run.smithi103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-19T13:14:11.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:11.900+0000 7fdaeb65d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-08-19T13:14:11.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:11.901+0000 7fdaeb65d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-08-19T13:14:11.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:11.901+0000 7fdaeb65d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-19T13:14:11.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:11.901+0000 7fdaeb65d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-19T13:14:11.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:11.901+0000 7fdaeb65d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-19T13:14:11.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:11.901+0000 7fdaeb65d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-08-19T13:14:14.743 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-08-19T13:14:14.769 DEBUG:teuthology.orchestra.run.smithi103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-19T13:14:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:14.854+0000 7f11ecfd8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-08-19T13:14:14.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:14.854+0000 7f11ecfd8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-08-19T13:14:14.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:14.854+0000 7f11ecfd8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-19T13:14:14.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:14.854+0000 7f11ecfd8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-19T13:14:14.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:14.854+0000 7f11ecfd8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-19T13:14:14.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:14.854+0000 7f11ecfd8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-08-19T13:14:17.700 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-08-19T13:14:17.729 DEBUG:teuthology.orchestra.run.smithi103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-19T13:14:17.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:17.816+0000 7fde1ae1e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-08-19T13:14:17.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:17.816+0000 7fde1ae1e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-08-19T13:14:17.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:17.816+0000 7fde1ae1e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-19T13:14:17.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:17.816+0000 7fde1ae1e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-19T13:14:17.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:17.816+0000 7fde1ae1e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-19T13:14:17.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:17.816+0000 7fde1ae1e640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-08-19T13:14:20.680 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-08-19T13:14:20.706 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-08-19T13:14:20.738 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-19T13:14:20.738 INFO:tasks.ceph:role: osd.4 2024-08-19T13:14:20.738 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com 2024-08-19T13:14:20.739 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-19T13:14:20.816 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-19T13:14:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-19T13:14:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-19T13:14:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-19T13:14:20.816 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-19T13:14:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-08-19T13:14:20.816 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-19T13:14:20.816 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-19T13:14:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-19T13:14:20.817 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-19T13:14:21.319 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-08-19T13:14:21.321 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-08-19T13:14:21.321 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-08-19T13:14:21.439 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-08-19T13:14:21.470 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-08-19T13:14:21.536 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-19T13:14:21.536 INFO:tasks.ceph:role: osd.5 2024-08-19T13:14:21.536 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com 2024-08-19T13:14:21.536 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-19T13:14:21.608 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-19T13:14:21.608 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-19T13:14:21.608 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-19T13:14:21.608 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-19T13:14:21.608 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-19T13:14:21.609 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-08-19T13:14:21.609 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-19T13:14:21.609 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-19T13:14:21.609 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-19T13:14:21.609 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-19T13:14:22.065 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-08-19T13:14:22.067 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-08-19T13:14:22.067 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-08-19T13:14:22.158 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-08-19T13:14:22.186 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-08-19T13:14:22.251 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-19T13:14:22.251 INFO:tasks.ceph:role: osd.6 2024-08-19T13:14:22.251 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com 2024-08-19T13:14:22.251 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-19T13:14:22.323 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-19T13:14:22.323 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-19T13:14:22.323 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-19T13:14:22.323 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-19T13:14:22.323 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-19T13:14:22.323 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-08-19T13:14:22.323 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-19T13:14:22.323 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-19T13:14:22.323 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-19T13:14:22.323 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-19T13:14:22.799 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-08-19T13:14:22.801 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-08-19T13:14:22.801 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-08-19T13:14:22.925 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-08-19T13:14:22.955 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-08-19T13:14:23.020 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-19T13:14:23.020 INFO:tasks.ceph:role: osd.7 2024-08-19T13:14:23.020 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi122.front.sepia.ceph.com 2024-08-19T13:14:23.020 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-08-19T13:14:23.092 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-08-19T13:14:23.093 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-19T13:14:23.093 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-19T13:14:23.093 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-19T13:14:23.093 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-19T13:14:23.093 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-08-19T13:14:23.093 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-19T13:14:23.093 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-19T13:14:23.093 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-19T13:14:23.093 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-19T13:14:23.601 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-08-19T13:14:23.603 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-08-19T13:14:23.603 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-08-19T13:14:23.727 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-08-19T13:14:23.757 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-19T13:14:23.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:23.854+0000 7fa539c1c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-08-19T13:14:23.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:23.854+0000 7fa539c1c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-08-19T13:14:23.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:23.854+0000 7fa539c1c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-19T13:14:23.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:23.854+0000 7fa539c1c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-19T13:14:23.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:23.854+0000 7fa539c1c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-19T13:14:23.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:23.855+0000 7fa539c1c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-08-19T13:14:26.707 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-08-19T13:14:26.738 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-19T13:14:26.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:26.831+0000 7fd249a1e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-08-19T13:14:26.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:26.831+0000 7fd249a1e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-08-19T13:14:26.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:26.831+0000 7fd249a1e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-19T13:14:26.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:26.831+0000 7fd249a1e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-19T13:14:26.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:26.831+0000 7fd249a1e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-19T13:14:26.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:26.832+0000 7fd249a1e640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-08-19T13:14:29.692 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-08-19T13:14:29.723 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-19T13:14:29.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:29.816+0000 7f7e14c21640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-08-19T13:14:29.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:29.816+0000 7f7e14c21640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-08-19T13:14:29.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:29.816+0000 7f7e14c21640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-19T13:14:29.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:29.816+0000 7f7e14c21640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-19T13:14:29.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:29.816+0000 7f7e14c21640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-19T13:14:29.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:29.816+0000 7f7e14c21640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-08-19T13:14:32.661 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-08-19T13:14:32.686 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-19T13:14:32.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:32.771+0000 7f68d54d5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-08-19T13:14:32.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:32.771+0000 7f68d54d5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-08-19T13:14:32.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:32.771+0000 7f68d54d5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-19T13:14:32.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:32.771+0000 7f68d54d5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-19T13:14:32.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:32.771+0000 7f68d54d5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-19T13:14:32.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:32.772+0000 7f68d54d5640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-08-19T13:14:35.631 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-08-19T13:14:35.661 INFO:tasks.ceph:Reading keys from all nodes... 2024-08-19T13:14:35.662 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:35.662 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-08-19T13:14:35.692 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:35.692 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-08-19T13:14:35.761 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:35.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-08-19T13:14:35.829 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:35.829 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-08-19T13:14:35.897 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:35.897 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-08-19T13:14:35.964 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:35.964 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-08-19T13:14:35.993 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:35.993 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-08-19T13:14:36.062 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:36.062 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-08-19T13:14:36.129 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:36.129 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-08-19T13:14:36.195 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:36.195 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-08-19T13:14:36.261 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:36.261 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-08-19T13:14:36.279 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:36.279 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-08-19T13:14:36.318 INFO:tasks.ceph:Adding keys to all mons... 2024-08-19T13:14:36.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-19T13:14:36.322 DEBUG:teuthology.orchestra.run.smithi122:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-19T13:14:36.362 INFO:teuthology.orchestra.run.smithi103.stdout:[mgr.y] 2024-08-19T13:14:36.362 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQAdRcNmFDIGIhAAj68NY1vOMsQrGKnzBpTLhg== 2024-08-19T13:14:36.362 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.0] 2024-08-19T13:14:36.362 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQAgRcNmO+rWNhAAmW4mYHce+lVijXZ7nZtroQ== 2024-08-19T13:14:36.362 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.1] 2024-08-19T13:14:36.362 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQAjRcNmEm+7NRAAa1xi860AGJVrIwKEj3E7gA== 2024-08-19T13:14:36.363 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.2] 2024-08-19T13:14:36.363 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQAmRcNmngb/MhAAvtBX0kAPpgTCuSDBS8Agbg== 2024-08-19T13:14:36.363 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.3] 2024-08-19T13:14:36.363 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQApRcNmk8G7MBAA7JxY9BFHwwuhO6lMNolgvg== 2024-08-19T13:14:36.363 INFO:teuthology.orchestra.run.smithi103.stdout:[mgr.x] 2024-08-19T13:14:36.363 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQAdRcNm99ztJRAAczwuh6g9d3rXzI3a/4UKuw== 2024-08-19T13:14:36.363 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.4] 2024-08-19T13:14:36.363 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQAvRcNmQ9cIMxAA5LxiV+khm3PuwhME7l/+KQ== 2024-08-19T13:14:36.363 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.5] 2024-08-19T13:14:36.363 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQAyRcNmf2OnMRAAep76dh+kJyBVSS2Zo9o3sA== 2024-08-19T13:14:36.363 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.6] 2024-08-19T13:14:36.363 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQA1RcNmIOG+MBAAyHHfH3Ws0587t/5ET3fkHA== 2024-08-19T13:14:36.363 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.7] 2024-08-19T13:14:36.364 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQA4RcNm1sgFLhAAh1dHQx+Lf/1XUndATkMEYA== 2024-08-19T13:14:36.364 INFO:teuthology.orchestra.run.smithi103.stdout:[client.0] 2024-08-19T13:14:36.364 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQAdRcNmBfyYKBAAldHJFBDnHkLYuEKJfeztmA== 2024-08-19T13:14:36.364 INFO:teuthology.orchestra.run.smithi103.stdout:[client.1] 2024-08-19T13:14:36.364 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQAdRcNmNkjEKxAAsmRLPzbjHdqdrY9uRuAFlQ== 2024-08-19T13:14:36.383 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.y] 2024-08-19T13:14:36.383 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAdRcNmFDIGIhAAj68NY1vOMsQrGKnzBpTLhg== 2024-08-19T13:14:36.383 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.0] 2024-08-19T13:14:36.383 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAgRcNmO+rWNhAAmW4mYHce+lVijXZ7nZtroQ== 2024-08-19T13:14:36.383 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.1] 2024-08-19T13:14:36.383 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAjRcNmEm+7NRAAa1xi860AGJVrIwKEj3E7gA== 2024-08-19T13:14:36.383 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.2] 2024-08-19T13:14:36.383 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAmRcNmngb/MhAAvtBX0kAPpgTCuSDBS8Agbg== 2024-08-19T13:14:36.383 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.3] 2024-08-19T13:14:36.383 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQApRcNmk8G7MBAA7JxY9BFHwwuhO6lMNolgvg== 2024-08-19T13:14:36.383 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.x] 2024-08-19T13:14:36.383 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAdRcNm99ztJRAAczwuh6g9d3rXzI3a/4UKuw== 2024-08-19T13:14:36.384 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.4] 2024-08-19T13:14:36.384 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAvRcNmQ9cIMxAA5LxiV+khm3PuwhME7l/+KQ== 2024-08-19T13:14:36.384 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.5] 2024-08-19T13:14:36.384 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAyRcNmf2OnMRAAep76dh+kJyBVSS2Zo9o3sA== 2024-08-19T13:14:36.384 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.6] 2024-08-19T13:14:36.384 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQA1RcNmIOG+MBAAyHHfH3Ws0587t/5ET3fkHA== 2024-08-19T13:14:36.384 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.7] 2024-08-19T13:14:36.384 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQA4RcNm1sgFLhAAh1dHQx+Lf/1XUndATkMEYA== 2024-08-19T13:14:36.384 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-08-19T13:14:36.384 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAdRcNmBfyYKBAAldHJFBDnHkLYuEKJfeztmA== 2024-08-19T13:14:36.384 INFO:teuthology.orchestra.run.smithi122.stdout:[client.1] 2024-08-19T13:14:36.384 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAdRcNmNkjEKxAAsmRLPzbjHdqdrY9uRuAFlQ== 2024-08-19T13:14:36.385 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-19T13:14:36.408 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-19T13:14:36.477 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.479 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.531 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.533 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.584 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.587 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.637 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.640 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-19T13:14:36.692 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-19T13:14:36.742 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.744 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.795 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.797 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.847 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.849 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.900 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.902 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T13:14:36.953 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-19T13:14:36.955 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-19T13:14:37.008 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-19T13:14:37.011 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-19T13:14:37.064 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-08-19T13:14:37.064 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-08-19T13:14:37.089 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-19T13:14:37.185 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-08-19T13:14:37.210 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-08-19T13:14:37.275 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-19T13:14:37.370 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-08-19T13:14:37.396 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-08-19T13:14:37.422 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-19T13:14:37.516 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-08-19T13:14:37.541 DEBUG:teuthology.orchestra.run.smithi103:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-19T13:14:37.543 DEBUG:teuthology.orchestra.run.smithi122:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-19T13:14:37.596 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-08-19T13:14:37.596 INFO:tasks.ceph.mon.a:Restarting daemon 2024-08-19T13:14:37.597 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-08-19T13:14:37.599 INFO:tasks.ceph.mon.a:Started 2024-08-19T13:14:37.599 INFO:tasks.ceph.mon.c:Restarting daemon 2024-08-19T13:14:37.599 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-08-19T13:14:37.601 INFO:tasks.ceph.mon.c:Started 2024-08-19T13:14:37.601 INFO:tasks.ceph.mon.b:Restarting daemon 2024-08-19T13:14:37.601 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-08-19T13:14:37.639 INFO:tasks.ceph.mon.b:Started 2024-08-19T13:14:37.639 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-08-19T13:14:37.639 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-08-19T13:14:37.640 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-08-19T13:14:37.642 INFO:tasks.ceph.mgr.y:Started 2024-08-19T13:14:37.642 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-08-19T13:14:37.642 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-08-19T13:14:37.644 INFO:tasks.ceph.mgr.x:Started 2024-08-19T13:14:37.645 INFO:tasks.ceph:Setting crush tunables to default 2024-08-19T13:14:37.645 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd crush tunables default 2024-08-19T13:14:37.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.743+0000 7f9d49b09640 1 Processor -- start 2024-08-19T13:14:37.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.743+0000 7f9d49b09640 1 -- start start 2024-08-19T13:14:37.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d49b09640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 0x7f9d44072c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:37.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d49b09640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:37.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d49b09640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d44073980 con 0x7f9d4413ba90 2024-08-19T13:14:37.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d49b09640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f9d440af8f0 con 0x7f9d4413be90 2024-08-19T13:14:37.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d49b09640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f9d4413f5f0 con 0x7f9d440af510 2024-08-19T13:14:37.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d43fff640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:37.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d43fff640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59794/0 (socket says 172.21.15.103:59794) 2024-08-19T13:14:37.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d43fff640 1 -- 172.21.15.103:0/1810007841 learned_addr learned my addr 172.21.15.103:0/1810007841 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:37.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d43fff640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 msgr2=0x7f9d4413f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T13:14:37.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d43fff640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 msgr2=0x7f9d4413f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T13:14:37.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:14:37.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 0x7f9d44072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:37.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T13:14:37.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 msgr2=0x7f9d44072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T13:14:37.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 msgr2=0x7f9d44072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T13:14:37.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 0x7f9d44072c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:14:37.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.744+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 0x7f9d44072c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T13:14:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.946+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.946+0000 7f9d43fff640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 msgr2=0x7f9d4413f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T13:14:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.946+0000 7f9d43fff640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 msgr2=0x7f9d4413f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T13:14:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.946+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:14:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.946+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T13:14:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.946+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 0x7f9d44072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.946+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 msgr2=0x7f9d44072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T13:14:37.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.946+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 msgr2=0x7f9d44072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T13:14:37.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.946+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 0x7f9d44072c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:14:37.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:37.946+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 0x7f9d44072c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T13:14:38.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:38.347+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:38.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:38.347+0000 7f9d43fff640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 msgr2=0x7f9d4413f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T13:14:38.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:38.347+0000 7f9d43fff640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 msgr2=0x7f9d4413f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T13:14:38.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:38.347+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:14:38.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:38.347+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T13:14:38.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:38.347+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 0x7f9d44072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:38.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:38.347+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 msgr2=0x7f9d44072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T13:14:38.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:38.347+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 msgr2=0x7f9d44072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T13:14:38.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:38.347+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 0x7f9d44072c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:14:38.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:38.347+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 0x7f9d44072c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T13:14:39.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:39.149+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:39.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:39.149+0000 7f9d43fff640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 msgr2=0x7f9d4413f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T13:14:39.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:39.149+0000 7f9d43fff640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 msgr2=0x7f9d4413f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T13:14:39.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:39.149+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:14:39.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:39.149+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T13:14:39.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:39.149+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 0x7f9d44072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:39.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:39.149+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 msgr2=0x7f9d44072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T13:14:39.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:39.149+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 msgr2=0x7f9d44072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T13:14:39.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:39.149+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 0x7f9d44072c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:14:39.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:39.149+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 0x7f9d44072c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T13:14:40.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.745+0000 7f9d417fa640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 msgr2=0x7f9d44072c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T13:14:40.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.745+0000 7f9d417fa640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.122:3300/0 conn(0x7f9d440af510 0x7f9d44072c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:40.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.745+0000 7f9d417fa640 1 -- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 msgr2=0x7f9d4413f0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T13:14:40.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.745+0000 7f9d417fa640 1 --2- 172.21.15.103:0/1810007841 >> v2:172.21.15.103:3300/0 conn(0x7f9d4413be90 0x7f9d4413f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:40.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.745+0000 7f9d417fa640 1 -- 172.21.15.103:0/1810007841 >> v1:172.21.15.103:6789/0 conn(0x7f9d4413ba90 legacy=0x7f9d4413e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:40.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.745+0000 7f9d417fa640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 0x7f9d30005b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:40.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.745+0000 7f9d417fa640 1 -- 172.21.15.103:0/1810007841 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d44071050 con 0x7f9d300061a0 2024-08-19T13:14:40.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.745+0000 7f9d417fa640 1 -- 172.21.15.103:0/1810007841 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9d440af8f0 con 0x7f9d300036c0 2024-08-19T13:14:40.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.745+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 0x7f9d30005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:40.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.746+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 msgr2=0x7f9d30005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T13:14:40.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.746+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 msgr2=0x7f9d30005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T13:14:40.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.746+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 0x7f9d30005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:14:40.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.746+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 0x7f9d30005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T13:14:40.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.947+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 0x7f9d30005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:40.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.947+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 msgr2=0x7f9d30005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T13:14:40.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.947+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 msgr2=0x7f9d30005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T13:14:40.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.947+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 0x7f9d30005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:14:40.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:40.947+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 0x7f9d30005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T13:14:41.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:41.348+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 0x7f9d30005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:41.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:41.348+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 msgr2=0x7f9d30005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T13:14:41.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:41.348+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 msgr2=0x7f9d30005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T13:14:41.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:41.348+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 0x7f9d30005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:14:41.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:41.348+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 0x7f9d30005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T13:14:42.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:42.149+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 0x7f9d30005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:42.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:42.149+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 msgr2=0x7f9d30005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T13:14:42.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:42.149+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 msgr2=0x7f9d30005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T13:14:42.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:42.149+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 0x7f9d30005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:14:42.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:42.149+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 0x7f9d30005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T13:14:43.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.746+0000 7f9d417fa640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 msgr2=0x7f9d30005b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T13:14:43.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.746+0000 7f9d417fa640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d300036c0 0x7f9d30005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:43.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.746+0000 7f9d417fa640 1 -- 172.21.15.103:0/1810007841 >> v1:172.21.15.122:6789/0 conn(0x7f9d300061a0 legacy=0x7f9d30008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:43.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.746+0000 7f9d43fff640 1 -- 172.21.15.103:0/1810007841 reap_dead start 2024-08-19T13:14:43.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.746+0000 7f9d417fa640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d30005b10 0x7f9d30009520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:43.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.746+0000 7f9d417fa640 1 -- 172.21.15.103:0/1810007841 --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d3000af40 con 0x7f9d4413ba90 2024-08-19T13:14:43.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.746+0000 7f9d417fa640 1 -- 172.21.15.103:0/1810007841 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d30009a60 con 0x7f9d3000a730 2024-08-19T13:14:43.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.746+0000 7f9d417fa640 1 -- 172.21.15.103:0/1810007841 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9d440af8f0 con 0x7f9d30005b10 2024-08-19T13:14:43.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.746+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d30005b10 0x7f9d30009520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:43.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.746+0000 7f9d43fff640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d30005b10 msgr2=0x7f9d30009520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T13:14:43.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.746+0000 7f9d43fff640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d30005b10 msgr2=0x7f9d30009520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T13:14:43.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.746+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d30005b10 0x7f9d30009520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:14:43.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.746+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d30005b10 0x7f9d30009520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T13:14:43.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.911+0000 7f9d427fc640 1 -- 172.21.15.103:0/1810007841 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2800379238 0 0) 0x7f9d3000af40 con 0x7f9d4413ba90 2024-08-19T13:14:43.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.911+0000 7f9d427fc640 1 -- 172.21.15.103:0/1810007841 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d2c003520 con 0x7f9d4413ba90 2024-08-19T13:14:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d427fc640 1 -- 172.21.15.103:0/1810007841 <== mon.0 v1:172.21.15.103:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2871218324 0 0) 0x7f9d2c003520 con 0x7f9d4413ba90 2024-08-19T13:14:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d427fc640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d30005b10 msgr2=0x7f9d30009520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T13:14:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d427fc640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d30005b10 0x7f9d30009520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d427fc640 1 -- 172.21.15.103:0/1810007841 >> v1:172.21.15.122:6789/0 conn(0x7f9d3000a730 legacy=0x7f9d30008e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d427fc640 1 -- 172.21.15.103:0/1810007841 --> v1:172.21.15.103:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d30009c30 con 0x7f9d4413ba90 2024-08-19T13:14:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d427fc640 1 -- 172.21.15.103:0/1810007841 <== mon.0 v1:172.21.15.103:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3066137496 0 0) 0x7f9d34015580 con 0x7f9d4413ba90 2024-08-19T13:14:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d427fc640 1 -- 172.21.15.103:0/1810007841 >> v1:172.21.15.103:6789/0 conn(0x7f9d4413ba90 legacy=0x7f9d3000a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d427fc640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d30005b10 0x7f9d2c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d427fc640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d2c004010 0x7f9d2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d427fc640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d2c004d40 0x7f9d2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d427fc640 1 -- 172.21.15.103:0/1810007841 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d440af8f0 con 0x7f9d2c004d40 2024-08-19T13:14:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d427fc640 1 -- 172.21.15.103:0/1810007841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c00bca0 con 0x7f9d2c004010 2024-08-19T13:14:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d427fc640 1 -- 172.21.15.103:0/1810007841 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2c00bea0 con 0x7f9d30005b10 2024-08-19T13:14:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d30005b10 0x7f9d2c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d2c004d40 0x7f9d2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.912+0000 7f9d42ffd640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d2c004010 0x7f9d2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.913+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d2c004010 msgr2=0x7f9d2c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.913+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d2c004010 0x7f9d2c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.913+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d2c004d40 msgr2=0x7f9d2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.913+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d2c004d40 0x7f9d2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:43.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.913+0000 7f9d437fe640 1 -- 172.21.15.103:0/1810007841 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c003520 con 0x7f9d30005b10 2024-08-19T13:14:43.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.913+0000 7f9d43fff640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d2c004d40 0x7f9d2c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:43.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.914+0000 7f9d437fe640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d30005b10 0x7f9d2c0039e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9d34006fd0 tx=0x7f9d2c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b228a401cbaf3284 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.914+0000 7f9d427fc640 1 -- 172.21.15.103:0/1810007841 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d34032070 con 0x7f9d30005b10 2024-08-19T13:14:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.914+0000 7f9d427fc640 1 -- 172.21.15.103:0/1810007841 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d34030070 con 0x7f9d30005b10 2024-08-19T13:14:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.914+0000 7f9d427fc640 1 -- 172.21.15.103:0/1810007841 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d34015d70 con 0x7f9d30005b10 2024-08-19T13:14:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.915+0000 7f9d49b09640 1 -- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d30005b10 msgr2=0x7f9d2c0039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.915+0000 7f9d49b09640 1 --2- 172.21.15.103:0/1810007841 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d30005b10 0x7f9d2c0039e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9d34006fd0 tx=0x7f9d2c005830 comp rx=0 tx=0).stop 2024-08-19T13:14:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.915+0000 7f9d43fff640 1 -- 172.21.15.103:0/1810007841 reap_dead start 2024-08-19T13:14:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.915+0000 7f9d49b09640 1 -- 172.21.15.103:0/1810007841 shutdown_connections 2024-08-19T13:14:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.915+0000 7f9d49b09640 1 -- 172.21.15.103:0/1810007841 >> 172.21.15.103:0/1810007841 conn(0x7f9d440b41c0 msgr2=0x7f9d440b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:43.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.915+0000 7f9d49b09640 1 -- 172.21.15.103:0/1810007841 shutdown_connections 2024-08-19T13:14:43.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.915+0000 7f9d49b09640 1 -- 172.21.15.103:0/1810007841 wait complete. 2024-08-19T13:14:43.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.916+0000 7f9d49b09640 1 Processor -- start 2024-08-19T13:14:43.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.917+0000 7f9d49b09640 1 -- start start 2024-08-19T13:14:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.917+0000 7f9d49b09640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d44141bf0 0x7f9d441d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.917+0000 7f9d49b09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d441d3ef0 0x7f9d440ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.917+0000 7f9d49b09640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d44138660 0x7f9d440adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.917+0000 7f9d49b09640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d44072ff0 con 0x7f9d44141bf0 2024-08-19T13:14:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.917+0000 7f9d49b09640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9d44072e70 con 0x7f9d441d3ef0 2024-08-19T13:14:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.917+0000 7f9d49b09640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9d440aea10 con 0x7f9d44138660 2024-08-19T13:14:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.918+0000 7f9d437fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d44141bf0 0x7f9d441d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.918+0000 7f9d437fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d44141bf0 0x7f9d441d39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59874/0 (socket says 172.21.15.103:59874) 2024-08-19T13:14:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.918+0000 7f9d437fe640 1 -- 172.21.15.103:0/2258019550 learned_addr learned my addr 172.21.15.103:0/2258019550 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.918+0000 7f9d42ffd640 1 --2- 172.21.15.103:0/2258019550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d441d3ef0 0x7f9d440ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.918+0000 7f9d43fff640 1 --2- 172.21.15.103:0/2258019550 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d44138660 0x7f9d440adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.918+0000 7f9d43fff640 1 -- 172.21.15.103:0/2258019550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d441d3ef0 msgr2=0x7f9d440ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.918+0000 7f9d43fff640 1 --2- 172.21.15.103:0/2258019550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d441d3ef0 0x7f9d440ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.918+0000 7f9d43fff640 1 -- 172.21.15.103:0/2258019550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d44141bf0 msgr2=0x7f9d441d39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.918+0000 7f9d43fff640 1 --2- 172.21.15.103:0/2258019550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d44141bf0 0x7f9d441d39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.918+0000 7f9d43fff640 1 -- 172.21.15.103:0/2258019550 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d440a9c60 con 0x7f9d44138660 2024-08-19T13:14:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.918+0000 7f9d42ffd640 1 --2- 172.21.15.103:0/2258019550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d441d3ef0 0x7f9d440ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.918+0000 7f9d43fff640 1 --2- 172.21.15.103:0/2258019550 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d44138660 0x7f9d440adc00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d440713b0 tx=0x7f9d3c019720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.919+0000 7f9d2bfff640 1 -- 172.21.15.103:0/2258019550 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00b910 con 0x7f9d44138660 2024-08-19T13:14:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.919+0000 7f9d2bfff640 1 -- 172.21.15.103:0/2258019550 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d3c003990 con 0x7f9d44138660 2024-08-19T13:14:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.919+0000 7f9d49b09640 1 -- 172.21.15.103:0/2258019550 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d44138140 con 0x7f9d44138660 2024-08-19T13:14:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.919+0000 7f9d2bfff640 1 -- 172.21.15.103:0/2258019550 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c003b80 con 0x7f9d44138660 2024-08-19T13:14:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.919+0000 7f9d49b09640 1 -- 172.21.15.103:0/2258019550 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d44137c10 con 0x7f9d44138660 2024-08-19T13:14:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.919+0000 7f9d2bfff640 1 -- 172.21.15.103:0/2258019550 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9d3c025020 con 0x7f9d44138660 2024-08-19T13:14:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.919+0000 7f9d2bfff640 1 -- 172.21.15.103:0/2258019550 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9d3c027960 con 0x7f9d44138660 2024-08-19T13:14:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.920+0000 7f9d49b09640 1 -- 172.21.15.103:0/2258019550 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d440aeda0 con 0x7f9d44138660 2024-08-19T13:14:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.923+0000 7f9d2bfff640 1 -- 172.21.15.103:0/2258019550 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f9d440a9c60 con 0x7f9d44138660 2024-08-19T13:14:43.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.989+0000 7f9d49b09640 1 -- 172.21.15.103:0/2258019550 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f9d44138310 con 0x7f9d44138660 2024-08-19T13:14:43.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.995+0000 7f9d2bfff640 1 -- 172.21.15.103:0/2258019550 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f9d3c027410 con 0x7f9d44138660 2024-08-19T13:14:43.996 INFO:teuthology.orchestra.run.smithi103.stderr:adjusted tunables profile to default 2024-08-19T13:14:43.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.996+0000 7f9d49b09640 1 -- 172.21.15.103:0/2258019550 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d44138660 msgr2=0x7f9d440adc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:43.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.996+0000 7f9d49b09640 1 --2- 172.21.15.103:0/2258019550 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d44138660 0x7f9d440adc00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d440713b0 tx=0x7f9d3c019720 comp rx=0 tx=0).stop 2024-08-19T13:14:43.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.996+0000 7f9d49b09640 1 -- 172.21.15.103:0/2258019550 shutdown_connections 2024-08-19T13:14:43.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.996+0000 7f9d49b09640 1 --2- 172.21.15.103:0/2258019550 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d44138660 0x7f9d440adc00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:43.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.996+0000 7f9d49b09640 1 --2- 172.21.15.103:0/2258019550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d441d3ef0 0x7f9d440ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:43.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.996+0000 7f9d49b09640 1 --2- 172.21.15.103:0/2258019550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d44141bf0 0x7f9d441d39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:43.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.996+0000 7f9d49b09640 1 -- 172.21.15.103:0/2258019550 >> 172.21.15.103:0/2258019550 conn(0x7f9d440b41c0 msgr2=0x7f9d44135450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:43.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.997+0000 7f9d49b09640 1 -- 172.21.15.103:0/2258019550 shutdown_connections 2024-08-19T13:14:43.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:43.997+0000 7f9d49b09640 1 -- 172.21.15.103:0/2258019550 wait complete. 2024-08-19T13:14:44.008 INFO:tasks.ceph:check_enable_crimson: False 2024-08-19T13:14:44.009 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-08-19T13:14:44.009 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:44.009 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-08-19T13:14:44.034 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:44.035 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-08-19T13:14:44.099 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:44.099 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-08-19T13:14:44.161 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:44.161+0000 7ffaf1e15040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-19T13:14:44.164 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:14:44.164 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-08-19T13:14:44.214 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:44.213+0000 7ffaf1e15040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-19T13:14:44.229 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:44.229 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-08-19T13:14:44.255 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:44.256 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-08-19T13:14:44.314 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:44.313+0000 7f3838329040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-19T13:14:44.320 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:44.321 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-08-19T13:14:44.322 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:44.322+0000 7ffaf1e15040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-19T13:14:44.366 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:44.365+0000 7f3838329040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-19T13:14:44.386 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-08-19T13:14:44.386 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-08-19T13:14:44.444 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:44.444+0000 7ffaf1e15040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-19T13:14:44.451 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 01d9bd23-e3e8-4905-af40-142b43a27628 0 2024-08-19T13:14:44.478 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:44.477+0000 7f3838329040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-19T13:14:44.556 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:44.556+0000 7ffaf1e15040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-19T13:14:44.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.574+0000 7f22b173d640 1 Processor -- start 2024-08-19T13:14:44.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.574+0000 7f22b173d640 1 -- start start 2024-08-19T13:14:44.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22b173d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22ac0b09b0 0x7f22ac06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22b173d640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f22ac146270 0x7f22ac14d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22b173d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22ac0715f0 con 0x7f22ac06d2f0 2024-08-19T13:14:44.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22b173d640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f22ac06d6d0 con 0x7f22ac146270 2024-08-19T13:14:44.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22b173d640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f22ac0718d0 con 0x7f22ac0b09b0 2024-08-19T13:14:44.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22aaffd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f22ac06d2f0 0x7f22ac145b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:33342/0 (socket says 172.21.15.122:33342) 2024-08-19T13:14:44.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22aaffd640 1 -- 172.21.15.122:0/3472318376 learned_addr learned my addr 172.21.15.122:0/3472318376 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-08-19T13:14:44.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22aa7fc640 1 --2- 172.21.15.122:0/3472318376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22ac0b09b0 0x7f22ac06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22ab7fe640 1 --2- 172.21.15.122:0/3472318376 >> v2:172.21.15.103:3300/0 conn(0x7f22ac146270 0x7f22ac14d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22a9ffb640 1 -- 172.21.15.122:0/3472318376 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 9392675 0 0) 0x7f22ac0715f0 con 0x7f22ac06d2f0 2024-08-19T13:14:44.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22a9ffb640 1 -- 172.21.15.122:0/3472318376 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2298003520 con 0x7f22ac06d2f0 2024-08-19T13:14:44.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22a9ffb640 1 -- 172.21.15.122:0/3472318376 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3508384859 0 0) 0x7f2298003520 con 0x7f22ac06d2f0 2024-08-19T13:14:44.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22a9ffb640 1 -- 172.21.15.122:0/3472318376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22ac0b09b0 msgr2=0x7f22ac06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:44.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22a9ffb640 1 --2- 172.21.15.122:0/3472318376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22ac0b09b0 0x7f22ac06e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22a9ffb640 1 -- 172.21.15.122:0/3472318376 >> v2:172.21.15.103:3300/0 conn(0x7f22ac146270 msgr2=0x7f22ac14d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:44.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22a9ffb640 1 --2- 172.21.15.122:0/3472318376 >> v2:172.21.15.103:3300/0 conn(0x7f22ac146270 0x7f22ac14d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22a9ffb640 1 -- 172.21.15.122:0/3472318376 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ac071bb0 con 0x7f22ac06d2f0 2024-08-19T13:14:44.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22a9ffb640 1 -- 172.21.15.122:0/3472318376 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3066137496 0 0) 0x7f229c001690 con 0x7f22ac06d2f0 2024-08-19T13:14:44.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22a9ffb640 1 -- 172.21.15.122:0/3472318376 >> v1:172.21.15.122:6789/0 conn(0x7f22ac06d2f0 legacy=0x7f22ac145b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:44.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.575+0000 7f22a9ffb640 1 --2- 172.21.15.122:0/3472318376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22980039b0 0x7f2298003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22a9ffb640 1 --2- 172.21.15.122:0/3472318376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2298004800 0x7f2298004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22a9ffb640 1 --2- 172.21.15.122:0/3472318376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22980051c0 0x7f229800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22a9ffb640 1 -- 172.21.15.122:0/3472318376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac06d6d0 con 0x7f22980039b0 2024-08-19T13:14:44.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22a9ffb640 1 -- 172.21.15.122:0/3472318376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac0718d0 con 0x7f22980051c0 2024-08-19T13:14:44.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22a9ffb640 1 -- 172.21.15.122:0/3472318376 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f229800bf50 con 0x7f2298004800 2024-08-19T13:14:44.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22aa7fc640 1 --2- 172.21.15.122:0/3472318376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22ac0b09b0 0x7f22ac06e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:44.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22ab7fe640 1 --2- 172.21.15.122:0/3472318376 >> v2:172.21.15.103:3300/0 conn(0x7f22ac146270 0x7f22ac14d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:44.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22aaffd640 1 --2- 172.21.15.122:0/3472318376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22980039b0 0x7f2298003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22ab7fe640 1 --2- 172.21.15.122:0/3472318376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22980051c0 0x7f229800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22aa7fc640 1 --2- 172.21.15.122:0/3472318376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2298004800 0x7f2298004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22aaffd640 1 -- 172.21.15.122:0/3472318376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2298004800 msgr2=0x7f2298004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:44.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22aaffd640 1 --2- 172.21.15.122:0/3472318376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2298004800 0x7f2298004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22aaffd640 1 -- 172.21.15.122:0/3472318376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22980051c0 msgr2=0x7f229800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:44.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22aaffd640 1 --2- 172.21.15.122:0/3472318376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22980051c0 0x7f229800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22aaffd640 1 -- 172.21.15.122:0/3472318376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2298003520 con 0x7f22980039b0 2024-08-19T13:14:44.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.576+0000 7f22ab7fe640 1 --2- 172.21.15.122:0/3472318376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22980051c0 0x7f229800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:44.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.577+0000 7f22aaffd640 1 --2- 172.21.15.122:0/3472318376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22980039b0 0x7f2298003dd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f229c00c210 tx=0x7f2298005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b190d34271868c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:44.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.577+0000 7f22a9ffb640 1 -- 172.21.15.122:0/3472318376 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c007d50 con 0x7f22980039b0 2024-08-19T13:14:44.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.577+0000 7f22a9ffb640 1 -- 172.21.15.122:0/3472318376 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f229c0058d0 con 0x7f22980039b0 2024-08-19T13:14:44.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.577+0000 7f22a9ffb640 1 -- 172.21.15.122:0/3472318376 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c005cc0 con 0x7f22980039b0 2024-08-19T13:14:44.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.577+0000 7f22b173d640 1 -- 172.21.15.122:0/3472318376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22980039b0 msgr2=0x7f2298003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:44.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.577+0000 7f22b173d640 1 --2- 172.21.15.122:0/3472318376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22980039b0 0x7f2298003dd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f229c00c210 tx=0x7f2298005830 comp rx=0 tx=0).stop 2024-08-19T13:14:44.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.577+0000 7f22ab7fe640 1 -- 172.21.15.122:0/3472318376 reap_dead start 2024-08-19T13:14:44.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.578+0000 7f22b173d640 1 -- 172.21.15.122:0/3472318376 shutdown_connections 2024-08-19T13:14:44.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.578+0000 7f22b173d640 1 -- 172.21.15.122:0/3472318376 >> 172.21.15.122:0/3472318376 conn(0x7f22ac0b5fb0 msgr2=0x7f22ac0b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:44.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.578+0000 7f22b173d640 1 -- 172.21.15.122:0/3472318376 shutdown_connections 2024-08-19T13:14:44.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.578+0000 7f22b173d640 1 -- 172.21.15.122:0/3472318376 wait complete. 2024-08-19T13:14:44.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.579+0000 7f22b173d640 1 Processor -- start 2024-08-19T13:14:44.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.579+0000 7f22b173d640 1 -- start start 2024-08-19T13:14:44.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.579+0000 7f22b173d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22980039b0 0x7f22ac145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.579+0000 7f22b173d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22ac0af660 0x7f22ac11db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.579+0000 7f22b173d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ac13c310 0x7f22ac13c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.579+0000 7f22b173d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac0b27f0 con 0x7f22ac13c310 2024-08-19T13:14:44.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.579+0000 7f22b173d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac073b20 con 0x7f22980039b0 2024-08-19T13:14:44.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.579+0000 7f22b173d640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f22ac0736f0 con 0x7f22ac0af660 2024-08-19T13:14:44.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.579+0000 7f22aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22980039b0 0x7f22ac145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.579+0000 7f22aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22980039b0 0x7f22ac145180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42676/0 (socket says 172.21.15.122:42676) 2024-08-19T13:14:44.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.579+0000 7f22aaffd640 1 -- 172.21.15.122:0/2087305314 learned_addr learned my addr 172.21.15.122:0/2087305314 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-08-19T13:14:44.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.579+0000 7f22aa7fc640 1 --2- 172.21.15.122:0/2087305314 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22ac0af660 0x7f22ac11db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.579+0000 7f22ab7fe640 1 --2- 172.21.15.122:0/2087305314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ac13c310 0x7f22ac13c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.580+0000 7f22aaffd640 1 -- 172.21.15.122:0/2087305314 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22ac0af660 msgr2=0x7f22ac11db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:44.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.580+0000 7f22aaffd640 1 --2- 172.21.15.122:0/2087305314 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22ac0af660 0x7f22ac11db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.580+0000 7f22aaffd640 1 -- 172.21.15.122:0/2087305314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ac13c310 msgr2=0x7f22ac13c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:44.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.580+0000 7f22aaffd640 1 --2- 172.21.15.122:0/2087305314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ac13c310 0x7f22ac13c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.580+0000 7f22aaffd640 1 -- 172.21.15.122:0/2087305314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ac13cce0 con 0x7f22980039b0 2024-08-19T13:14:44.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.580+0000 7f22aaffd640 1 --2- 172.21.15.122:0/2087305314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22980039b0 0x7f22ac145180 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f229c02ae30 tx=0x7f22ac0717c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:44.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.580+0000 7f22a37fe640 1 -- 172.21.15.122:0/2087305314 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c007d50 con 0x7f22980039b0 2024-08-19T13:14:44.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.580+0000 7f22a37fe640 1 -- 172.21.15.122:0/2087305314 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f229c041070 con 0x7f22980039b0 2024-08-19T13:14:44.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.580+0000 7f22b173d640 1 -- 172.21.15.122:0/2087305314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ac1db010 con 0x7f22980039b0 2024-08-19T13:14:44.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.580+0000 7f22a37fe640 1 -- 172.21.15.122:0/2087305314 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c001c90 con 0x7f22980039b0 2024-08-19T13:14:44.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.580+0000 7f22b173d640 1 -- 172.21.15.122:0/2087305314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ac1db550 con 0x7f22980039b0 2024-08-19T13:14:44.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.580+0000 7f22a37fe640 1 -- 172.21.15.122:0/2087305314 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f229c03b070 con 0x7f22980039b0 2024-08-19T13:14:44.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.580+0000 7f22a37fe640 1 -- 172.21.15.122:0/2087305314 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f229c033a40 con 0x7f22980039b0 2024-08-19T13:14:44.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.581+0000 7f22b173d640 1 -- 172.21.15.122:0/2087305314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ac11f770 con 0x7f22980039b0 2024-08-19T13:14:44.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.585+0000 7f22a37fe640 1 -- 172.21.15.122:0/2087305314 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f229c033420 con 0x7f22980039b0 2024-08-19T13:14:44.608 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:44.607+0000 7f3838329040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-19T13:14:44.651 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:44.650+0000 7ffaf1e15040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-19T13:14:44.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.654+0000 7f22b173d640 1 -- 172.21.15.122:0/2087305314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "01d9bd23-e3e8-4905-af40-142b43a27628", "id": 0} v 0) -- 0x7f22ac146700 con 0x7f22980039b0 2024-08-19T13:14:44.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.660+0000 7f22a37fe640 1 -- 172.21.15.122:0/2087305314 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "01d9bd23-e3e8-4905-af40-142b43a27628", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f229c033c90 con 0x7f22980039b0 2024-08-19T13:14:44.661 INFO:teuthology.orchestra.run.smithi122.stdout:0 2024-08-19T13:14:44.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.661+0000 7f22b173d640 1 -- 172.21.15.122:0/2087305314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22980039b0 msgr2=0x7f22ac145180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:44.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.661+0000 7f22b173d640 1 --2- 172.21.15.122:0/2087305314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22980039b0 0x7f22ac145180 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f229c02ae30 tx=0x7f22ac0717c0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.661+0000 7f22b173d640 1 -- 172.21.15.122:0/2087305314 shutdown_connections 2024-08-19T13:14:44.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.661+0000 7f22b173d640 1 --2- 172.21.15.122:0/2087305314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ac13c310 0x7f22ac13c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.661+0000 7f22b173d640 1 --2- 172.21.15.122:0/2087305314 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22ac0af660 0x7f22ac11db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.661+0000 7f22b173d640 1 --2- 172.21.15.122:0/2087305314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22980039b0 0x7f22ac145180 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.661+0000 7f22b173d640 1 -- 172.21.15.122:0/2087305314 >> 172.21.15.122:0/2087305314 conn(0x7f22ac0b5fb0 msgr2=0x7f22ac071df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:44.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.661+0000 7f22b173d640 1 -- 172.21.15.122:0/2087305314 shutdown_connections 2024-08-19T13:14:44.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.661+0000 7f22b173d640 1 -- 172.21.15.122:0/2087305314 wait complete. 2024-08-19T13:14:44.672 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new d562d5f9-b0ac-4f9c-856d-cdcf1be3da18 1 2024-08-19T13:14:44.728 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:44.727+0000 7f3838329040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-19T13:14:44.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.768+0000 7f03dd0ec640 1 Processor -- start 2024-08-19T13:14:44.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.768+0000 7f03dd0ec640 1 -- start start 2024-08-19T13:14:44.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03dd0ec640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f03d80709b0 0x7f03d8070d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03dd0ec640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f03d813f290 0x7f03d813f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03dd0ec640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03d80af470 con 0x7f03d8071a70 2024-08-19T13:14:44.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03dd0ec640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f03d8073300 con 0x7f03d813f290 2024-08-19T13:14:44.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03dd0ec640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f03d813fbe0 con 0x7f03d80709b0 2024-08-19T13:14:44.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03d6d9d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f03d80709b0 0x7f03d8070d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03d659c640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f03d8071a70 0x7f03d8073890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:33358/0 (socket says 172.21.15.122:33358) 2024-08-19T13:14:44.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03d659c640 1 -- 172.21.15.122:0/2510627078 learned_addr learned my addr 172.21.15.122:0/2510627078 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-08-19T13:14:44.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03d6d9d640 1 --2- 172.21.15.122:0/2510627078 >> v2:172.21.15.122:3300/0 conn(0x7f03d80709b0 0x7f03d8070d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42682/0 (socket says 172.21.15.122:42682) 2024-08-19T13:14:44.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03d759e640 1 --2- 172.21.15.122:0/2510627078 >> v2:172.21.15.103:3300/0 conn(0x7f03d813f290 0x7f03d813f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03d6d9d640 1 -- 172.21.15.122:0/2510627078 >> v2:172.21.15.103:3300/0 conn(0x7f03d813f290 msgr2=0x7f03d813f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:44.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03d6d9d640 1 --2- 172.21.15.122:0/2510627078 >> v2:172.21.15.103:3300/0 conn(0x7f03d813f290 0x7f03d813f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03d5d9b640 1 -- 172.21.15.122:0/2510627078 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3682498024 0 0) 0x7f03d80af470 con 0x7f03d8071a70 2024-08-19T13:14:44.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03d6d9d640 1 -- 172.21.15.122:0/2510627078 >> v1:172.21.15.122:6789/0 conn(0x7f03d8071a70 legacy=0x7f03d8073890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:44.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.769+0000 7f03d6d9d640 1 -- 172.21.15.122:0/2510627078 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d813fd90 con 0x7f03d80709b0 2024-08-19T13:14:44.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.770+0000 7f03d6d9d640 1 --2- 172.21.15.122:0/2510627078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03d80709b0 0x7f03d8070d90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f03d80730f0 tx=0x7f03c802c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae61a87861731c65 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:44.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.771+0000 7f03d5d9b640 1 -- 172.21.15.122:0/2510627078 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8032770 con 0x7f03d80709b0 2024-08-19T13:14:44.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.771+0000 7f03d5d9b640 1 -- 172.21.15.122:0/2510627078 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c8038070 con 0x7f03d80709b0 2024-08-19T13:14:44.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.771+0000 7f03d5d9b640 1 -- 172.21.15.122:0/2510627078 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8032b30 con 0x7f03d80709b0 2024-08-19T13:14:44.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.771+0000 7f03dd0ec640 1 -- 172.21.15.122:0/2510627078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03d80709b0 msgr2=0x7f03d8070d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:44.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.771+0000 7f03dd0ec640 1 --2- 172.21.15.122:0/2510627078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03d80709b0 0x7f03d8070d90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f03d80730f0 tx=0x7f03c802c870 comp rx=0 tx=0).stop 2024-08-19T13:14:44.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.771+0000 7f03dd0ec640 1 -- 172.21.15.122:0/2510627078 shutdown_connections 2024-08-19T13:14:44.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.771+0000 7f03dd0ec640 1 --2- 172.21.15.122:0/2510627078 >> v2:172.21.15.103:3300/0 conn(0x7f03d813f290 0x7f03d813f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.771+0000 7f03dd0ec640 1 --2- 172.21.15.122:0/2510627078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03d80709b0 0x7f03d8070d90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.771+0000 7f03dd0ec640 1 -- 172.21.15.122:0/2510627078 >> 172.21.15.122:0/2510627078 conn(0x7f03d80b5030 msgr2=0x7f03d80b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:44.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.771+0000 7f03dd0ec640 1 -- 172.21.15.122:0/2510627078 shutdown_connections 2024-08-19T13:14:44.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.771+0000 7f03dd0ec640 1 -- 172.21.15.122:0/2510627078 wait complete. 2024-08-19T13:14:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.773+0000 7f03dd0ec640 1 Processor -- start 2024-08-19T13:14:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.773+0000 7f03dd0ec640 1 -- start start 2024-08-19T13:14:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.773+0000 7f03dd0ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03d80709b0 0x7f03d8142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.773+0000 7f03dd0ec640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03d813f290 0x7f03d80ab190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.773+0000 7f03dd0ec640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03d80ab770 0x7f03d8139040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.773+0000 7f03dd0ec640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03d806b980 con 0x7f03d80709b0 2024-08-19T13:14:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.773+0000 7f03dd0ec640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03d806b800 con 0x7f03d813f290 2024-08-19T13:14:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.773+0000 7f03dd0ec640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f03d806bb00 con 0x7f03d80ab770 2024-08-19T13:14:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.773+0000 7f03d659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03d813f290 0x7f03d80ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.773+0000 7f03d659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03d813f290 0x7f03d80ab190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42686/0 (socket says 172.21.15.122:42686) 2024-08-19T13:14:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.773+0000 7f03d659c640 1 -- 172.21.15.122:0/346482595 learned_addr learned my addr 172.21.15.122:0/346482595 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-08-19T13:14:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.773+0000 7f03d6d9d640 1 --2- 172.21.15.122:0/346482595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03d80709b0 0x7f03d8142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.773+0000 7f03d759e640 1 --2- 172.21.15.122:0/346482595 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03d80ab770 0x7f03d8139040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.774+0000 7f03d659c640 1 -- 172.21.15.122:0/346482595 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03d80ab770 msgr2=0x7f03d8139040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:44.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.774+0000 7f03d659c640 1 --2- 172.21.15.122:0/346482595 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03d80ab770 0x7f03d8139040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.774+0000 7f03d659c640 1 -- 172.21.15.122:0/346482595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03d80709b0 msgr2=0x7f03d8142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:44.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.774+0000 7f03d659c640 1 --2- 172.21.15.122:0/346482595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03d80709b0 0x7f03d8142900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.774+0000 7f03d659c640 1 -- 172.21.15.122:0/346482595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d80ac120 con 0x7f03d813f290 2024-08-19T13:14:44.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.774+0000 7f03d659c640 1 --2- 172.21.15.122:0/346482595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03d813f290 0x7f03d80ab190 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00c4d0 tx=0x7f03cc00c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:44.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.774+0000 7f03bf7fe640 1 -- 172.21.15.122:0/346482595 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc018070 con 0x7f03d813f290 2024-08-19T13:14:44.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.774+0000 7f03bf7fe640 1 -- 172.21.15.122:0/346482595 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03cc007500 con 0x7f03d813f290 2024-08-19T13:14:44.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.774+0000 7f03dd0ec640 1 -- 172.21.15.122:0/346482595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d80ad370 con 0x7f03d813f290 2024-08-19T13:14:44.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.774+0000 7f03bf7fe640 1 -- 172.21.15.122:0/346482595 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc004a20 con 0x7f03d813f290 2024-08-19T13:14:44.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.774+0000 7f03dd0ec640 1 -- 172.21.15.122:0/346482595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d8139760 con 0x7f03d813f290 2024-08-19T13:14:44.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.774+0000 7f03bf7fe640 1 -- 172.21.15.122:0/346482595 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f03cc01b070 con 0x7f03d813f290 2024-08-19T13:14:44.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.774+0000 7f03bf7fe640 1 -- 172.21.15.122:0/346482595 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f03cc0127b0 con 0x7f03d813f290 2024-08-19T13:14:44.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.775+0000 7f03dd0ec640 1 -- 172.21.15.122:0/346482595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d8070d90 con 0x7f03d813f290 2024-08-19T13:14:44.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.777+0000 7f03bf7fe640 1 -- 172.21.15.122:0/346482595 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f03d80ad370 con 0x7f03d813f290 2024-08-19T13:14:44.799 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:44.798+0000 7ffaf1e15040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-19T13:14:44.826 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:44.825+0000 7f3838329040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-19T13:14:44.840 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:44.840+0000 7ffaf1e15040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-19T13:14:44.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.844+0000 7f03dd0ec640 1 -- 172.21.15.122:0/346482595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d562d5f9-b0ac-4f9c-856d-cdcf1be3da18", "id": 1} v 0) -- 0x7f03d80abdc0 con 0x7f03d813f290 2024-08-19T13:14:44.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.849+0000 7f03bf7fe640 1 -- 172.21.15.122:0/346482595 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d562d5f9-b0ac-4f9c-856d-cdcf1be3da18", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f03cc01d440 con 0x7f03d813f290 2024-08-19T13:14:44.849 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-08-19T13:14:44.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.850+0000 7f03dd0ec640 1 -- 172.21.15.122:0/346482595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03d813f290 msgr2=0x7f03d80ab190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:44.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.850+0000 7f03dd0ec640 1 --2- 172.21.15.122:0/346482595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03d813f290 0x7f03d80ab190 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00c4d0 tx=0x7f03cc00c990 comp rx=0 tx=0).stop 2024-08-19T13:14:44.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.850+0000 7f03dd0ec640 1 -- 172.21.15.122:0/346482595 shutdown_connections 2024-08-19T13:14:44.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.850+0000 7f03dd0ec640 1 --2- 172.21.15.122:0/346482595 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03d80ab770 0x7f03d8139040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.850+0000 7f03dd0ec640 1 --2- 172.21.15.122:0/346482595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03d813f290 0x7f03d80ab190 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.850+0000 7f03dd0ec640 1 --2- 172.21.15.122:0/346482595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03d80709b0 0x7f03d8142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.850+0000 7f03dd0ec640 1 -- 172.21.15.122:0/346482595 >> 172.21.15.122:0/346482595 conn(0x7f03d80b5030 msgr2=0x7f03d8139ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:44.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.850+0000 7f03dd0ec640 1 -- 172.21.15.122:0/346482595 shutdown_connections 2024-08-19T13:14:44.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.850+0000 7f03dd0ec640 1 -- 172.21.15.122:0/346482595 wait complete. 2024-08-19T13:14:44.861 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new adc47b95-8fbe-48c0-bb44-f78ba8ed6d29 2 2024-08-19T13:14:44.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.958+0000 7fc5b8d09640 1 Processor -- start 2024-08-19T13:14:44.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.958+0000 7fc5b8d09640 1 -- start start 2024-08-19T13:14:44.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.959+0000 7fc5b8d09640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5b40afca0 0x7fc5b406b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.959+0000 7fc5b8d09640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc5b4145ed0 0x7fc5b4146360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.959+0000 7fc5b8d09640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5b40af570 con 0x7fc5b40af8c0 2024-08-19T13:14:44.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.959+0000 7fc5b8d09640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc5b406bd40 con 0x7fc5b4145ed0 2024-08-19T13:14:44.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.959+0000 7fc5b8d09640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b4145a40 con 0x7fc5b40afca0 2024-08-19T13:14:44.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.959+0000 7fc5b2d9d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc5b4145ed0 0x7fc5b4146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.959+0000 7fc5b2d9d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc5b4145ed0 0x7fc5b4146360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42688/0 (socket says 172.21.15.122:42688) 2024-08-19T13:14:44.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.959+0000 7fc5b2d9d640 1 -- 172.21.15.122:0/3289085908 learned_addr learned my addr 172.21.15.122:0/3289085908 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-08-19T13:14:44.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.959+0000 7fc5b1d9b640 1 --2- 172.21.15.122:0/3289085908 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5b40afca0 0x7fc5b406b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.960+0000 7fc5b2d9d640 1 -- 172.21.15.122:0/3289085908 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5b40afca0 msgr2=0x7fc5b406b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:44.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.960+0000 7fc5b2d9d640 1 --2- 172.21.15.122:0/3289085908 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5b40afca0 0x7fc5b406b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.960+0000 7fc5b2d9d640 1 -- 172.21.15.122:0/3289085908 >> v1:172.21.15.103:6789/0 conn(0x7fc5b40af8c0 legacy=0x7fc5b40ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:44.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.960+0000 7fc5b2d9d640 1 -- 172.21.15.122:0/3289085908 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b40ae860 con 0x7fc5b4145ed0 2024-08-19T13:14:44.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.960+0000 7fc5b1d9b640 1 --2- 172.21.15.122:0/3289085908 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5b40afca0 0x7fc5b406b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T13:14:44.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.960+0000 7fc5b2d9d640 1 --2- 172.21.15.122:0/3289085908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b4145ed0 0x7fc5b4146360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009e50 tx=0x7fc5ac02a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edc73ce9afc6de08 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:44.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.960+0000 7fc5b159a640 1 -- 172.21.15.122:0/3289085908 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac036070 con 0x7fc5b4145ed0 2024-08-19T13:14:44.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.960+0000 7fc5b159a640 1 -- 172.21.15.122:0/3289085908 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5ac031600 con 0x7fc5b4145ed0 2024-08-19T13:14:44.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.960+0000 7fc5b159a640 1 -- 172.21.15.122:0/3289085908 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac0319d0 con 0x7fc5b4145ed0 2024-08-19T13:14:44.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.961+0000 7fc5b8d09640 1 -- 172.21.15.122:0/3289085908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b4145ed0 msgr2=0x7fc5b4146360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:44.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.961+0000 7fc5b8d09640 1 --2- 172.21.15.122:0/3289085908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b4145ed0 0x7fc5b4146360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009e50 tx=0x7fc5ac02a630 comp rx=0 tx=0).stop 2024-08-19T13:14:44.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.961+0000 7fc5b8d09640 1 -- 172.21.15.122:0/3289085908 shutdown_connections 2024-08-19T13:14:44.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.961+0000 7fc5b8d09640 1 --2- 172.21.15.122:0/3289085908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b4145ed0 0x7fc5b4146360 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.961+0000 7fc5b8d09640 1 --2- 172.21.15.122:0/3289085908 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5b40afca0 0x7fc5b406b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.961+0000 7fc5b8d09640 1 -- 172.21.15.122:0/3289085908 >> 172.21.15.122:0/3289085908 conn(0x7fc5b40b50a0 msgr2=0x7fc5b40b54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.961+0000 7fc5b8d09640 1 -- 172.21.15.122:0/3289085908 shutdown_connections 2024-08-19T13:14:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.961+0000 7fc5b8d09640 1 -- 172.21.15.122:0/3289085908 wait complete. 2024-08-19T13:14:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.962+0000 7fc5b8d09640 1 Processor -- start 2024-08-19T13:14:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.962+0000 7fc5b8d09640 1 -- start start 2024-08-19T13:14:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b8d09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b40af8c0 0x7fc5b413f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b8d09640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5b40afca0 0x7fc5b4140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b8d09640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5b4145ed0 0x7fc5b4140880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b8d09640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b40b0790 con 0x7fc5b40afca0 2024-08-19T13:14:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b8d09640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b40b0610 con 0x7fc5b40af8c0 2024-08-19T13:14:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b8d09640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b414cdb0 con 0x7fc5b4145ed0 2024-08-19T13:14:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b40af8c0 0x7fc5b413f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b40af8c0 0x7fc5b413f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42692/0 (socket says 172.21.15.122:42692) 2024-08-19T13:14:44.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b259c640 1 -- 172.21.15.122:0/3667583143 learned_addr learned my addr 172.21.15.122:0/3667583143 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-08-19T13:14:44.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b1d9b640 1 --2- 172.21.15.122:0/3667583143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5b40afca0 0x7fc5b4140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b2d9d640 1 --2- 172.21.15.122:0/3667583143 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5b4145ed0 0x7fc5b4140880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:44.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b259c640 1 -- 172.21.15.122:0/3667583143 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5b4145ed0 msgr2=0x7fc5b4140880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:44.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b259c640 1 --2- 172.21.15.122:0/3667583143 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5b4145ed0 0x7fc5b4140880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b259c640 1 -- 172.21.15.122:0/3667583143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5b40afca0 msgr2=0x7fc5b4140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:44.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b259c640 1 --2- 172.21.15.122:0/3667583143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5b40afca0 0x7fc5b4140340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:44.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b259c640 1 -- 172.21.15.122:0/3667583143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b4071050 con 0x7fc5b40af8c0 2024-08-19T13:14:44.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5b259c640 1 --2- 172.21.15.122:0/3667583143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b40af8c0 0x7fc5b413f080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400c430 tx=0x7fc5a400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:44.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5a2ffd640 1 -- 172.21.15.122:0/3667583143 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4011790 con 0x7fc5b40af8c0 2024-08-19T13:14:44.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.963+0000 7fc5a2ffd640 1 -- 172.21.15.122:0/3667583143 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a4017070 con 0x7fc5b40af8c0 2024-08-19T13:14:44.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.964+0000 7fc5a2ffd640 1 -- 172.21.15.122:0/3667583143 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4011b70 con 0x7fc5b40af8c0 2024-08-19T13:14:44.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.964+0000 7fc5b8d09640 1 -- 172.21.15.122:0/3667583143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b40701a0 con 0x7fc5b40af8c0 2024-08-19T13:14:44.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.964+0000 7fc5b8d09640 1 -- 172.21.15.122:0/3667583143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b4142810 con 0x7fc5b40af8c0 2024-08-19T13:14:44.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.964+0000 7fc5a2ffd640 1 -- 172.21.15.122:0/3667583143 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc5a4011d10 con 0x7fc5b40af8c0 2024-08-19T13:14:44.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.964+0000 7fc5a2ffd640 1 -- 172.21.15.122:0/3667583143 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fc5a4012b00 con 0x7fc5b40af8c0 2024-08-19T13:14:44.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.964+0000 7fc5b8d09640 1 -- 172.21.15.122:0/3667583143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b406dc70 con 0x7fc5b40af8c0 2024-08-19T13:14:44.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:44.966+0000 7fc5a2ffd640 1 -- 172.21.15.122:0/3667583143 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7fc5b406dc70 con 0x7fc5b40af8c0 2024-08-19T13:14:44.980 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:44.979+0000 7f3838329040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-19T13:14:45.022 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:45.021+0000 7f3838329040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-19T13:14:45.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.035+0000 7fc5b8d09640 1 -- 172.21.15.122:0/3667583143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "adc47b95-8fbe-48c0-bb44-f78ba8ed6d29", "id": 2} v 0) -- 0x7fc5b4143c20 con 0x7fc5b40af8c0 2024-08-19T13:14:45.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.043+0000 7fc5a2ffd640 1 -- 172.21.15.122:0/3667583143 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "adc47b95-8fbe-48c0-bb44-f78ba8ed6d29", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fc5a40126a0 con 0x7fc5b40af8c0 2024-08-19T13:14:45.043 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-08-19T13:14:45.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.044+0000 7fc5b8d09640 1 -- 172.21.15.122:0/3667583143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b40af8c0 msgr2=0x7fc5b413f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.044+0000 7fc5b8d09640 1 --2- 172.21.15.122:0/3667583143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b40af8c0 0x7fc5b413f080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400c430 tx=0x7fc5a400c8f0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.044+0000 7fc5b8d09640 1 -- 172.21.15.122:0/3667583143 shutdown_connections 2024-08-19T13:14:45.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.044+0000 7fc5b8d09640 1 --2- 172.21.15.122:0/3667583143 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5b4145ed0 0x7fc5b4140880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.044+0000 7fc5b8d09640 1 --2- 172.21.15.122:0/3667583143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5b40afca0 0x7fc5b4140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.044+0000 7fc5b8d09640 1 --2- 172.21.15.122:0/3667583143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b40af8c0 0x7fc5b413f080 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.044+0000 7fc5b8d09640 1 -- 172.21.15.122:0/3667583143 >> 172.21.15.122:0/3667583143 conn(0x7fc5b40b50a0 msgr2=0x7fc5b4144f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:45.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.044+0000 7fc5b8d09640 1 -- 172.21.15.122:0/3667583143 shutdown_connections 2024-08-19T13:14:45.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.044+0000 7fc5b8d09640 1 -- 172.21.15.122:0/3667583143 wait complete. 2024-08-19T13:14:45.059 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new a2e730c7-32c8-4387-bdee-e00a70e7613a 3 2024-08-19T13:14:45.136 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:45.135+0000 7ffaf1e15040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-19T13:14:45.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.155+0000 7f57c76a7640 1 Processor -- start 2024-08-19T13:14:45.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.155+0000 7f57c76a7640 1 -- start start 2024-08-19T13:14:45.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.156+0000 7f57c76a7640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f57c0073430 0x7f57c013f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.156+0000 7f57c76a7640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f57c00af3d0 0x7f57c0070870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.156+0000 7f57c76a7640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f57c0070db0 0x7f57c013f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.156+0000 7f57c76a7640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f57c00729c0 con 0x7f57c00af3d0 2024-08-19T13:14:45.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.156+0000 7f57c76a7640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f57c0072ca0 con 0x7f57c0073430 2024-08-19T13:14:45.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.156+0000 7f57c76a7640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f57c0073810 con 0x7f57c0070db0 2024-08-19T13:14:45.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.156+0000 7f57c5443640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f57c0073430 0x7f57c013f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.156+0000 7f57c5443640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f57c0073430 0x7f57c013f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42706/0 (socket says 172.21.15.122:42706) 2024-08-19T13:14:45.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.156+0000 7f57c5443640 1 -- 172.21.15.122:0/780395118 learned_addr learned my addr 172.21.15.122:0/780395118 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-08-19T13:14:45.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.157+0000 7f57c4c42640 1 --2- 172.21.15.122:0/780395118 >> v2:172.21.15.103:3300/0 conn(0x7f57c00af3d0 0x7f57c0070870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.157+0000 7f57c5c44640 1 --2- 172.21.15.122:0/780395118 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f57c0070db0 0x7f57c013f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.157+0000 7f57c5443640 1 -- 172.21.15.122:0/780395118 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f57c0070db0 msgr2=0x7f57c013f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:45.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.157+0000 7f57c5443640 1 --2- 172.21.15.122:0/780395118 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f57c0070db0 0x7f57c013f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.157+0000 7f57c5443640 1 -- 172.21.15.122:0/780395118 >> v2:172.21.15.103:3300/0 conn(0x7f57c00af3d0 msgr2=0x7f57c0070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:45.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.157+0000 7f57c5443640 1 --2- 172.21.15.122:0/780395118 >> v2:172.21.15.103:3300/0 conn(0x7f57c00af3d0 0x7f57c0070870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.157+0000 7f57c5443640 1 -- 172.21.15.122:0/780395118 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c0071850 con 0x7f57c0073430 2024-08-19T13:14:45.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.157+0000 7f57c5443640 1 --2- 172.21.15.122:0/780395118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c0073430 0x7f57c013f270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f57c0070010 tx=0x7f57b402a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4f622eec30a337e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:45.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.158+0000 7f57b3fff640 1 -- 172.21.15.122:0/780395118 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4037070 con 0x7f57c0073430 2024-08-19T13:14:45.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.158+0000 7f57b3fff640 1 -- 172.21.15.122:0/780395118 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b402ae00 con 0x7f57c0073430 2024-08-19T13:14:45.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.158+0000 7f57b3fff640 1 -- 172.21.15.122:0/780395118 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4032660 con 0x7f57c0073430 2024-08-19T13:14:45.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.158+0000 7f57c76a7640 1 -- 172.21.15.122:0/780395118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c0073430 msgr2=0x7f57c013f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.158+0000 7f57c76a7640 1 --2- 172.21.15.122:0/780395118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c0073430 0x7f57c013f270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f57c0070010 tx=0x7f57b402a0b0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.159+0000 7f57c76a7640 1 -- 172.21.15.122:0/780395118 shutdown_connections 2024-08-19T13:14:45.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.159+0000 7f57c76a7640 1 --2- 172.21.15.122:0/780395118 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f57c0070db0 0x7f57c013f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.159+0000 7f57c76a7640 1 --2- 172.21.15.122:0/780395118 >> v2:172.21.15.103:3300/0 conn(0x7f57c00af3d0 0x7f57c0070870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.159+0000 7f57c76a7640 1 --2- 172.21.15.122:0/780395118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c0073430 0x7f57c013f270 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.159+0000 7f57c76a7640 1 -- 172.21.15.122:0/780395118 >> 172.21.15.122:0/780395118 conn(0x7f57c00b50a0 msgr2=0x7f57c00b54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:45.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.159+0000 7f57c76a7640 1 -- 172.21.15.122:0/780395118 shutdown_connections 2024-08-19T13:14:45.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.159+0000 7f57c76a7640 1 -- 172.21.15.122:0/780395118 wait complete. 2024-08-19T13:14:45.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.160+0000 7f57c76a7640 1 Processor -- start 2024-08-19T13:14:45.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.160+0000 7f57c76a7640 1 -- start start 2024-08-19T13:14:45.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.161+0000 7f57c76a7640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f57c0073430 0x7f57c0138af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.161+0000 7f57c76a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57c0141a50 0x7f57c0140600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.161+0000 7f57c76a7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c0140b40 0x7f57c006b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.161+0000 7f57c76a7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01401c0 con 0x7f57c0141a50 2024-08-19T13:14:45.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.161+0000 7f57c76a7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f57c0140040 con 0x7f57c0140b40 2024-08-19T13:14:45.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.161+0000 7f57c76a7640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f57c0142770 con 0x7f57c0073430 2024-08-19T13:14:45.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.161+0000 7f57c5c44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c0140b40 0x7f57c006b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.161+0000 7f57c5c44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c0140b40 0x7f57c006b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42720/0 (socket says 172.21.15.122:42720) 2024-08-19T13:14:45.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.161+0000 7f57c5c44640 1 -- 172.21.15.122:0/90600277 learned_addr learned my addr 172.21.15.122:0/90600277 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-08-19T13:14:45.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.161+0000 7f57c5443640 1 --2- 172.21.15.122:0/90600277 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f57c0073430 0x7f57c0138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.161+0000 7f57c4c42640 1 --2- 172.21.15.122:0/90600277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57c0141a50 0x7f57c0140600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.162+0000 7f57c5c44640 1 -- 172.21.15.122:0/90600277 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f57c0073430 msgr2=0x7f57c0138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.162+0000 7f57c5c44640 1 --2- 172.21.15.122:0/90600277 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f57c0073430 0x7f57c0138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.162+0000 7f57c5c44640 1 -- 172.21.15.122:0/90600277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57c0141a50 msgr2=0x7f57c0140600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.162+0000 7f57c5c44640 1 --2- 172.21.15.122:0/90600277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57c0141a50 0x7f57c0140600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.162+0000 7f57c5c44640 1 -- 172.21.15.122:0/90600277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c0141f60 con 0x7f57c0140b40 2024-08-19T13:14:45.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.162+0000 7f57c4c42640 1 --2- 172.21.15.122:0/90600277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57c0141a50 0x7f57c0140600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:45.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.162+0000 7f57c5c44640 1 --2- 172.21.15.122:0/90600277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c0140b40 0x7f57c006b800 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f57b800eea0 tx=0x7f57b800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:45.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.162+0000 7f57b1ffb640 1 -- 172.21.15.122:0/90600277 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b8018020 con 0x7f57c0140b40 2024-08-19T13:14:45.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.162+0000 7f57b1ffb640 1 -- 172.21.15.122:0/90600277 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b8014400 con 0x7f57c0140b40 2024-08-19T13:14:45.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.162+0000 7f57c76a7640 1 -- 172.21.15.122:0/90600277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c0142240 con 0x7f57c0140b40 2024-08-19T13:14:45.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.162+0000 7f57b1ffb640 1 -- 172.21.15.122:0/90600277 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b80147f0 con 0x7f57c0140b40 2024-08-19T13:14:45.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.162+0000 7f57c76a7640 1 -- 172.21.15.122:0/90600277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c00ae4d0 con 0x7f57c0140b40 2024-08-19T13:14:45.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.162+0000 7f57b1ffb640 1 -- 172.21.15.122:0/90600277 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f57b8005d10 con 0x7f57c0140b40 2024-08-19T13:14:45.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.162+0000 7f57b1ffb640 1 -- 172.21.15.122:0/90600277 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f57b801dde0 con 0x7f57c0140b40 2024-08-19T13:14:45.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.163+0000 7f57c76a7640 1 -- 172.21.15.122:0/90600277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c013f270 con 0x7f57c0140b40 2024-08-19T13:14:45.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.165+0000 7f57b1ffb640 1 -- 172.21.15.122:0/90600277 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f57b8022030 con 0x7f57c0140b40 2024-08-19T13:14:45.182 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:45.182+0000 7ffaf1e15040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-19T13:14:45.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.232+0000 7f57c76a7640 1 -- 172.21.15.122:0/90600277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a2e730c7-32c8-4387-bdee-e00a70e7613a", "id": 3} v 0) -- 0x7f57c00b17a0 con 0x7f57c0140b40 2024-08-19T13:14:45.235 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:45.235+0000 7ffaf1e15040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-19T13:14:45.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.240+0000 7f57b1ffb640 1 -- 172.21.15.122:0/90600277 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a2e730c7-32c8-4387-bdee-e00a70e7613a", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f57b801d950 con 0x7f57c0140b40 2024-08-19T13:14:45.241 INFO:teuthology.orchestra.run.smithi122.stdout:3 2024-08-19T13:14:45.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.241+0000 7f57c76a7640 1 -- 172.21.15.122:0/90600277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c0140b40 msgr2=0x7f57c006b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.241+0000 7f57c76a7640 1 --2- 172.21.15.122:0/90600277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c0140b40 0x7f57c006b800 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f57b800eea0 tx=0x7f57b800ef90 comp rx=0 tx=0).stop 2024-08-19T13:14:45.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.241+0000 7f57c76a7640 1 -- 172.21.15.122:0/90600277 shutdown_connections 2024-08-19T13:14:45.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.241+0000 7f57c76a7640 1 --2- 172.21.15.122:0/90600277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c0140b40 0x7f57c006b800 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.241+0000 7f57c76a7640 1 --2- 172.21.15.122:0/90600277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57c0141a50 0x7f57c0140600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.241+0000 7f57c76a7640 1 --2- 172.21.15.122:0/90600277 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f57c0073430 0x7f57c0138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.241+0000 7f57c76a7640 1 -- 172.21.15.122:0/90600277 >> 172.21.15.122:0/90600277 conn(0x7f57c00b50a0 msgr2=0x7f57c00b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:45.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.241+0000 7f57c76a7640 1 -- 172.21.15.122:0/90600277 shutdown_connections 2024-08-19T13:14:45.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.241+0000 7f57c76a7640 1 -- 172.21.15.122:0/90600277 wait complete. 2024-08-19T13:14:45.252 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 45b5502e-ce4c-411c-b7a2-1742483a432f 4 2024-08-19T13:14:45.317 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:45.316+0000 7f3838329040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-19T13:14:45.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.349+0000 7f055a4c8640 1 Processor -- start 2024-08-19T13:14:45.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.349+0000 7f055a4c8640 1 -- start start 2024-08-19T13:14:45.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.349+0000 7f055a4c8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0554073430 0x7f055413f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.349+0000 7f055a4c8640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f05540af3d0 0x7f0554070870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.349+0000 7f055a4c8640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05540729c0 con 0x7f0554070db0 2024-08-19T13:14:45.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.349+0000 7f055a4c8640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f0554072ca0 con 0x7f05540af3d0 2024-08-19T13:14:45.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.349+0000 7f055a4c8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f0554073810 con 0x7f0554073430 2024-08-19T13:14:45.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.349+0000 7f0553fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0554073430 0x7f055413f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.349+0000 7f0553fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0554073430 0x7f055413f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42722/0 (socket says 172.21.15.122:42722) 2024-08-19T13:14:45.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.350+0000 7f0553fff640 1 -- 172.21.15.122:0/645567948 learned_addr learned my addr 172.21.15.122:0/645567948 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-08-19T13:14:45.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.350+0000 7f05537fe640 1 --2- 172.21.15.122:0/645567948 >> v2:172.21.15.103:3300/0 conn(0x7f05540af3d0 0x7f0554070870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.350+0000 7f0553fff640 1 -- 172.21.15.122:0/645567948 >> v2:172.21.15.103:3300/0 conn(0x7f05540af3d0 msgr2=0x7f0554070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:45.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.350+0000 7f0553fff640 1 --2- 172.21.15.122:0/645567948 >> v2:172.21.15.103:3300/0 conn(0x7f05540af3d0 0x7f0554070870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.350+0000 7f0553fff640 1 -- 172.21.15.122:0/645567948 >> v1:172.21.15.122:6789/0 conn(0x7f0554070db0 legacy=0x7f055413f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.350+0000 7f0553fff640 1 -- 172.21.15.122:0/645567948 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0554071850 con 0x7f0554073430 2024-08-19T13:14:45.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.350+0000 7f0552ffd640 1 -- 172.21.15.122:0/645567948 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1487322279 0 0) 0x7f05540729c0 con 0x7f0554070db0 2024-08-19T13:14:45.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.350+0000 7f0553fff640 1 --2- 172.21.15.122:0/645567948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0554073430 0x7f055413f270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0554070010 tx=0x7f054402a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b7c8155c82cd31e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:45.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.351+0000 7f0552ffd640 1 -- 172.21.15.122:0/645567948 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544037070 con 0x7f0554073430 2024-08-19T13:14:45.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.351+0000 7f0552ffd640 1 -- 172.21.15.122:0/645567948 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0544032440 con 0x7f0554073430 2024-08-19T13:14:45.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.351+0000 7f0552ffd640 1 -- 172.21.15.122:0/645567948 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544032810 con 0x7f0554073430 2024-08-19T13:14:45.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.351+0000 7f055a4c8640 1 -- 172.21.15.122:0/645567948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0554073430 msgr2=0x7f055413f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.351+0000 7f055a4c8640 1 --2- 172.21.15.122:0/645567948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0554073430 0x7f055413f270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0554070010 tx=0x7f054402a150 comp rx=0 tx=0).stop 2024-08-19T13:14:45.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.351+0000 7f055a4c8640 1 -- 172.21.15.122:0/645567948 shutdown_connections 2024-08-19T13:14:45.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.351+0000 7f055a4c8640 1 --2- 172.21.15.122:0/645567948 >> v2:172.21.15.103:3300/0 conn(0x7f05540af3d0 0x7f0554070870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.351+0000 7f055a4c8640 1 --2- 172.21.15.122:0/645567948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0554073430 0x7f055413f270 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.351+0000 7f055a4c8640 1 -- 172.21.15.122:0/645567948 >> 172.21.15.122:0/645567948 conn(0x7f05540b50a0 msgr2=0x7f05540b54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:45.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.351+0000 7f055a4c8640 1 -- 172.21.15.122:0/645567948 shutdown_connections 2024-08-19T13:14:45.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.351+0000 7f055a4c8640 1 -- 172.21.15.122:0/645567948 wait complete. 2024-08-19T13:14:45.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.353+0000 7f055a4c8640 1 Processor -- start 2024-08-19T13:14:45.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.353+0000 7f055a4c8640 1 -- start start 2024-08-19T13:14:45.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f055a4c8640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0554073430 0x7f055413ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f055a4c8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f055413c560 0x7f0554141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f055a4c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0554142480 0x7f05541d7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f055a4c8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0554149c40 con 0x7f0554142480 2024-08-19T13:14:45.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f055a4c8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0554149ac0 con 0x7f055413c560 2024-08-19T13:14:45.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f055a4c8640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0554149dc0 con 0x7f0554073430 2024-08-19T13:14:45.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f05537fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f055413c560 0x7f0554141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f05537fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f055413c560 0x7f0554141f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42726/0 (socket says 172.21.15.122:42726) 2024-08-19T13:14:45.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f05537fe640 1 -- 172.21.15.122:0/3418195113 learned_addr learned my addr 172.21.15.122:0/3418195113 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-08-19T13:14:45.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f0553fff640 1 --2- 172.21.15.122:0/3418195113 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0554073430 0x7f055413ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f0558a65640 1 --2- 172.21.15.122:0/3418195113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0554142480 0x7f05541d7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f05537fe640 1 -- 172.21.15.122:0/3418195113 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0554073430 msgr2=0x7f055413ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f05537fe640 1 --2- 172.21.15.122:0/3418195113 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0554073430 0x7f055413ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f05537fe640 1 -- 172.21.15.122:0/3418195113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0554142480 msgr2=0x7f05541d7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f05537fe640 1 --2- 172.21.15.122:0/3418195113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0554142480 0x7f05541d7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.354+0000 7f05537fe640 1 -- 172.21.15.122:0/3418195113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0554140ea0 con 0x7f055413c560 2024-08-19T13:14:45.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.355+0000 7f05537fe640 1 --2- 172.21.15.122:0/3418195113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f055413c560 0x7f0554141f40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0548010ab0 tx=0x7f0548010f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:45.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.355+0000 7f0550ff9640 1 -- 172.21.15.122:0/3418195113 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054801e070 con 0x7f055413c560 2024-08-19T13:14:45.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.355+0000 7f0550ff9640 1 -- 172.21.15.122:0/3418195113 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0548011450 con 0x7f055413c560 2024-08-19T13:14:45.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.355+0000 7f0550ff9640 1 -- 172.21.15.122:0/3418195113 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548011840 con 0x7f055413c560 2024-08-19T13:14:45.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.355+0000 7f055a4c8640 1 -- 172.21.15.122:0/3418195113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0554141180 con 0x7f055413c560 2024-08-19T13:14:45.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.355+0000 7f055a4c8640 1 -- 172.21.15.122:0/3418195113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055413ca40 con 0x7f055413c560 2024-08-19T13:14:45.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.355+0000 7f0550ff9640 1 -- 172.21.15.122:0/3418195113 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0548021070 con 0x7f055413c560 2024-08-19T13:14:45.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.355+0000 7f0550ff9640 1 -- 172.21.15.122:0/3418195113 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f0548023e00 con 0x7f055413c560 2024-08-19T13:14:45.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.356+0000 7f055a4c8640 1 -- 172.21.15.122:0/3418195113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055413f270 con 0x7f055413c560 2024-08-19T13:14:45.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.358+0000 7f0550ff9640 1 -- 172.21.15.122:0/3418195113 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f0548023440 con 0x7f055413c560 2024-08-19T13:14:45.368 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:45.368+0000 7ffaf1e15040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-19T13:14:45.372 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:45.371+0000 7f3838329040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-19T13:14:45.413 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:45.412+0000 7ffaf1e15040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-19T13:14:45.428 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:45.427+0000 7f3838329040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-19T13:14:45.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.435+0000 7f055a4c8640 1 -- 172.21.15.122:0/3418195113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "45b5502e-ce4c-411c-b7a2-1742483a432f", "id": 4} v 0) -- 0x7f05540b4390 con 0x7f055413c560 2024-08-19T13:14:45.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.443+0000 7f0550ff9640 1 -- 172.21.15.122:0/3418195113 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "45b5502e-ce4c-411c-b7a2-1742483a432f", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f05480239c0 con 0x7f055413c560 2024-08-19T13:14:45.444 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-08-19T13:14:45.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.444+0000 7f055a4c8640 1 -- 172.21.15.122:0/3418195113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f055413c560 msgr2=0x7f0554141f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.444+0000 7f055a4c8640 1 --2- 172.21.15.122:0/3418195113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f055413c560 0x7f0554141f40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0548010ab0 tx=0x7f0548010f70 comp rx=0 tx=0).stop 2024-08-19T13:14:45.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.444+0000 7f055a4c8640 1 -- 172.21.15.122:0/3418195113 shutdown_connections 2024-08-19T13:14:45.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.444+0000 7f055a4c8640 1 --2- 172.21.15.122:0/3418195113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0554142480 0x7f05541d7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.444+0000 7f055a4c8640 1 --2- 172.21.15.122:0/3418195113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f055413c560 0x7f0554141f40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.444+0000 7f055a4c8640 1 --2- 172.21.15.122:0/3418195113 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0554073430 0x7f055413ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.444+0000 7f055a4c8640 1 -- 172.21.15.122:0/3418195113 >> 172.21.15.122:0/3418195113 conn(0x7f05540b50a0 msgr2=0x7f05540b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:45.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.444+0000 7f055a4c8640 1 -- 172.21.15.122:0/3418195113 shutdown_connections 2024-08-19T13:14:45.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.444+0000 7f055a4c8640 1 -- 172.21.15.122:0/3418195113 wait complete. 2024-08-19T13:14:45.455 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 056397bb-673f-461f-87c8-e44390d7ba37 5 2024-08-19T13:14:45.464 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:45.464+0000 7ffaf1e15040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-19T13:14:45.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.552+0000 7f80d1cc1640 1 Processor -- start 2024-08-19T13:14:45.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.552+0000 7f80d1cc1640 1 -- start start 2024-08-19T13:14:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.553+0000 7f80d1cc1640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f80cc073430 0x7f80cc13f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.553+0000 7f80d1cc1640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f80cc0af3d0 0x7f80cc070870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.553+0000 7f80d1cc1640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80cc0729c0 con 0x7f80cc070db0 2024-08-19T13:14:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.553+0000 7f80d1cc1640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f80cc072ca0 con 0x7f80cc073430 2024-08-19T13:14:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.553+0000 7f80d1cc1640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f80cc073810 con 0x7f80cc0af3d0 2024-08-19T13:14:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.553+0000 7f80caffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f80cc0af3d0 0x7f80cc070870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.553+0000 7f80caffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f80cc0af3d0 0x7f80cc070870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42740/0 (socket says 172.21.15.122:42740) 2024-08-19T13:14:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.553+0000 7f80caffd640 1 -- 172.21.15.122:0/4267094434 learned_addr learned my addr 172.21.15.122:0/4267094434 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-08-19T13:14:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.553+0000 7f80cbfff640 1 --1- 172.21.15.122:0/4267094434 >> v1:172.21.15.122:6789/0 conn(0x7f80cc070db0 0x7f80cc13f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:33364/0 (socket says 172.21.15.122:33364) 2024-08-19T13:14:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.553+0000 7f80cb7fe640 1 --2- 172.21.15.122:0/4267094434 >> v2:172.21.15.103:3300/0 conn(0x7f80cc073430 0x7f80cc13f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.553+0000 7f80caffd640 1 -- 172.21.15.122:0/4267094434 >> v2:172.21.15.103:3300/0 conn(0x7f80cc073430 msgr2=0x7f80cc13f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:45.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.553+0000 7f80caffd640 1 --2- 172.21.15.122:0/4267094434 >> v2:172.21.15.103:3300/0 conn(0x7f80cc073430 0x7f80cc13f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.553+0000 7f80caffd640 1 -- 172.21.15.122:0/4267094434 >> v1:172.21.15.122:6789/0 conn(0x7f80cc070db0 legacy=0x7f80cc13f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.553+0000 7f80caffd640 1 -- 172.21.15.122:0/4267094434 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc071850 con 0x7f80cc0af3d0 2024-08-19T13:14:45.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.554+0000 7f80caffd640 1 --2- 172.21.15.122:0/4267094434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80cc0af3d0 0x7f80cc070870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f80cc072590 tx=0x7f80b402a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9faf6182679a6690 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:45.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.554+0000 7f80ca7fc640 1 -- 172.21.15.122:0/4267094434 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4037070 con 0x7f80cc0af3d0 2024-08-19T13:14:45.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.554+0000 7f80ca7fc640 1 -- 172.21.15.122:0/4267094434 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b4032440 con 0x7f80cc0af3d0 2024-08-19T13:14:45.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.554+0000 7f80ca7fc640 1 -- 172.21.15.122:0/4267094434 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4032810 con 0x7f80cc0af3d0 2024-08-19T13:14:45.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.555+0000 7f80d1cc1640 1 -- 172.21.15.122:0/4267094434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80cc0af3d0 msgr2=0x7f80cc070870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.555+0000 7f80d1cc1640 1 --2- 172.21.15.122:0/4267094434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80cc0af3d0 0x7f80cc070870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f80cc072590 tx=0x7f80b402a150 comp rx=0 tx=0).stop 2024-08-19T13:14:45.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.555+0000 7f80d1cc1640 1 -- 172.21.15.122:0/4267094434 shutdown_connections 2024-08-19T13:14:45.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.555+0000 7f80d1cc1640 1 --2- 172.21.15.122:0/4267094434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80cc0af3d0 0x7f80cc070870 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.555+0000 7f80d1cc1640 1 --2- 172.21.15.122:0/4267094434 >> v2:172.21.15.103:3300/0 conn(0x7f80cc073430 0x7f80cc13f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.555+0000 7f80d1cc1640 1 -- 172.21.15.122:0/4267094434 >> 172.21.15.122:0/4267094434 conn(0x7f80cc0b50a0 msgr2=0x7f80cc0b54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:45.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.555+0000 7f80d1cc1640 1 -- 172.21.15.122:0/4267094434 shutdown_connections 2024-08-19T13:14:45.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.555+0000 7f80d1cc1640 1 -- 172.21.15.122:0/4267094434 wait complete. 2024-08-19T13:14:45.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.556+0000 7f80d1cc1640 1 Processor -- start 2024-08-19T13:14:45.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.556+0000 7f80d1cc1640 1 -- start start 2024-08-19T13:14:45.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.556+0000 7f80d1cc1640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80cc073430 0x7f80cc0adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.556+0000 7f80d1cc1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80cc1db030 0x7f80cc1ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.556+0000 7f80d1cc1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80cc1cad30 0x7f80cc1cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.556+0000 7f80d1cc1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc06b980 con 0x7f80cc1db030 2024-08-19T13:14:45.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.556+0000 7f80d1cc1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc06b800 con 0x7f80cc1cad30 2024-08-19T13:14:45.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.556+0000 7f80d1cc1640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f80cc06bb00 con 0x7f80cc073430 2024-08-19T13:14:45.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80cbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80cc1cad30 0x7f80cc1cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80cbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80cc1cad30 0x7f80cc1cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42756/0 (socket says 172.21.15.122:42756) 2024-08-19T13:14:45.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80caffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80cc1db030 0x7f80cc1ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80cbfff640 1 -- 172.21.15.122:0/751545601 learned_addr learned my addr 172.21.15.122:0/751545601 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-08-19T13:14:45.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80cb7fe640 1 --2- 172.21.15.122:0/751545601 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80cc073430 0x7f80cc0adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80cbfff640 1 -- 172.21.15.122:0/751545601 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80cc073430 msgr2=0x7f80cc0adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80cbfff640 1 --2- 172.21.15.122:0/751545601 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80cc073430 0x7f80cc0adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80cbfff640 1 -- 172.21.15.122:0/751545601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80cc1db030 msgr2=0x7f80cc1ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80cbfff640 1 --2- 172.21.15.122:0/751545601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80cc1db030 0x7f80cc1ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80cbfff640 1 -- 172.21.15.122:0/751545601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc140df0 con 0x7f80cc1cad30 2024-08-19T13:14:45.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80caffd640 1 --2- 172.21.15.122:0/751545601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80cc1db030 0x7f80cc1ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:45.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80cb7fe640 1 --2- 172.21.15.122:0/751545601 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80cc073430 0x7f80cc0adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:45.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80cbfff640 1 --2- 172.21.15.122:0/751545601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80cc1cad30 0x7f80cc1cb1e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f80c400c7a0 tx=0x7f80c400cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:45.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80c3fff640 1 -- 172.21.15.122:0/751545601 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4017070 con 0x7f80cc1cad30 2024-08-19T13:14:45.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80c3fff640 1 -- 172.21.15.122:0/751545601 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c400a640 con 0x7f80cc1cad30 2024-08-19T13:14:45.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80c3fff640 1 -- 172.21.15.122:0/751545601 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c400a9f0 con 0x7f80cc1cad30 2024-08-19T13:14:45.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80d1cc1640 1 -- 172.21.15.122:0/751545601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80cc141060 con 0x7f80cc1cad30 2024-08-19T13:14:45.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80d1cc1640 1 -- 172.21.15.122:0/751545601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80cc1d7b10 con 0x7f80cc1cad30 2024-08-19T13:14:45.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.557+0000 7f80c3fff640 1 -- 172.21.15.122:0/751545601 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f80c401a070 con 0x7f80cc1cad30 2024-08-19T13:14:45.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.558+0000 7f80c3fff640 1 -- 172.21.15.122:0/751545601 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f80c40077f0 con 0x7f80cc1cad30 2024-08-19T13:14:45.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.558+0000 7f80d1cc1640 1 -- 172.21.15.122:0/751545601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80cc13f270 con 0x7f80cc1cad30 2024-08-19T13:14:45.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.560+0000 7f80c3fff640 1 -- 172.21.15.122:0/751545601 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f80c400ab90 con 0x7f80cc1cad30 2024-08-19T13:14:45.568 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:45.567+0000 7f3838329040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-19T13:14:45.613 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:45.613+0000 7f3838329040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-19T13:14:45.614 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:45.614+0000 7ffaf1e15040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-19T13:14:45.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.628+0000 7f80d1cc1640 1 -- 172.21.15.122:0/751545601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "056397bb-673f-461f-87c8-e44390d7ba37", "id": 5} v 0) -- 0x7f80cc0b17a0 con 0x7f80cc1cad30 2024-08-19T13:14:45.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.633+0000 7f80c3fff640 1 -- 172.21.15.122:0/751545601 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "056397bb-673f-461f-87c8-e44390d7ba37", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f80c400ad70 con 0x7f80cc1cad30 2024-08-19T13:14:45.634 INFO:teuthology.orchestra.run.smithi122.stdout:5 2024-08-19T13:14:45.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.633+0000 7f80d1cc1640 1 -- 172.21.15.122:0/751545601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80cc1cad30 msgr2=0x7f80cc1cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.633+0000 7f80d1cc1640 1 --2- 172.21.15.122:0/751545601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80cc1cad30 0x7f80cc1cb1e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f80c400c7a0 tx=0x7f80c400cc60 comp rx=0 tx=0).stop 2024-08-19T13:14:45.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.633+0000 7f80d1cc1640 1 -- 172.21.15.122:0/751545601 shutdown_connections 2024-08-19T13:14:45.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.633+0000 7f80d1cc1640 1 --2- 172.21.15.122:0/751545601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80cc1cad30 0x7f80cc1cb1e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.633+0000 7f80d1cc1640 1 --2- 172.21.15.122:0/751545601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80cc1db030 0x7f80cc1ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.633+0000 7f80d1cc1640 1 --2- 172.21.15.122:0/751545601 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80cc073430 0x7f80cc0adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.634+0000 7f80d1cc1640 1 -- 172.21.15.122:0/751545601 >> 172.21.15.122:0/751545601 conn(0x7f80cc0b50a0 msgr2=0x7f80cc13a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:45.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.634+0000 7f80d1cc1640 1 -- 172.21.15.122:0/751545601 shutdown_connections 2024-08-19T13:14:45.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.634+0000 7f80d1cc1640 1 -- 172.21.15.122:0/751545601 wait complete. 2024-08-19T13:14:45.645 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new fab2d2d2-c670-4f56-b099-ca4686395179 6 2024-08-19T13:14:45.667 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:45.666+0000 7f3838329040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-19T13:14:45.671 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:45.671+0000 7ffaf1e15040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-19T13:14:45.716 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:45.715+0000 7ffaf1e15040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-19T13:14:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.742+0000 7f45c2f44640 1 Processor -- start 2024-08-19T13:14:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.742+0000 7f45c2f44640 1 -- start start 2024-08-19T13:14:45.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.743+0000 7f45c2f44640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f45bc071c20 0x7f45bc0af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.743+0000 7f45c2f44640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45bc072d50 0x7f45bc0af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.743+0000 7f45c2f44640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45bc073e40 con 0x7f45bc1403f0 2024-08-19T13:14:45.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.743+0000 7f45c2f44640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f45bc072560 con 0x7f45bc071c20 2024-08-19T13:14:45.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.743+0000 7f45c2f44640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc0b0620 con 0x7f45bc072d50 2024-08-19T13:14:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.743+0000 7f45bbfff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45bc072d50 0x7f45bc0af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.743+0000 7f45c0ce0640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f45bc071c20 0x7f45bc0af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.743+0000 7f45c14e1640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f45bc1403f0 0x7f45bc1408a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.122:56880/0 (socket says 172.21.15.122:56880) 2024-08-19T13:14:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.743+0000 7f45c14e1640 1 -- 172.21.15.122:0/4265505335 learned_addr learned my addr 172.21.15.122:0/4265505335 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-08-19T13:14:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.744+0000 7f45c0ce0640 1 -- 172.21.15.122:0/4265505335 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45bc072d50 msgr2=0x7f45bc0af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.744+0000 7f45c0ce0640 1 --2- 172.21.15.122:0/4265505335 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45bc072d50 0x7f45bc0af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.744+0000 7f45c0ce0640 1 -- 172.21.15.122:0/4265505335 >> v1:172.21.15.103:6789/0 conn(0x7f45bc1403f0 legacy=0x7f45bc1408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.744+0000 7f45c0ce0640 1 -- 172.21.15.122:0/4265505335 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc072000 con 0x7f45bc071c20 2024-08-19T13:14:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.744+0000 7f45bb7fe640 1 -- 172.21.15.122:0/4265505335 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 949974631 0 0) 0x7f45bc073e40 con 0x7f45bc1403f0 2024-08-19T13:14:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.744+0000 7f45c0ce0640 1 --2- 172.21.15.122:0/4265505335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45bc071c20 0x7f45bc0af000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f45b00097a0 tx=0x7f45b002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa06b681b569c56 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.744+0000 7f45bb7fe640 1 -- 172.21.15.122:0/4265505335 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0037070 con 0x7f45bc071c20 2024-08-19T13:14:45.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.745+0000 7f45bb7fe640 1 -- 172.21.15.122:0/4265505335 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b0032440 con 0x7f45bc071c20 2024-08-19T13:14:45.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.745+0000 7f45bb7fe640 1 -- 172.21.15.122:0/4265505335 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0032810 con 0x7f45bc071c20 2024-08-19T13:14:45.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.745+0000 7f45c2f44640 1 -- 172.21.15.122:0/4265505335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45bc071c20 msgr2=0x7f45bc0af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.745+0000 7f45c2f44640 1 --2- 172.21.15.122:0/4265505335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45bc071c20 0x7f45bc0af000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f45b00097a0 tx=0x7f45b002a180 comp rx=0 tx=0).stop 2024-08-19T13:14:45.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.745+0000 7f45c2f44640 1 -- 172.21.15.122:0/4265505335 shutdown_connections 2024-08-19T13:14:45.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.745+0000 7f45c2f44640 1 --2- 172.21.15.122:0/4265505335 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45bc072d50 0x7f45bc0af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.745+0000 7f45c2f44640 1 --2- 172.21.15.122:0/4265505335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45bc071c20 0x7f45bc0af000 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.745+0000 7f45c2f44640 1 -- 172.21.15.122:0/4265505335 >> 172.21.15.122:0/4265505335 conn(0x7f45bc0b6330 msgr2=0x7f45bc0b6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:45.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.745+0000 7f45c2f44640 1 -- 172.21.15.122:0/4265505335 shutdown_connections 2024-08-19T13:14:45.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.745+0000 7f45c2f44640 1 -- 172.21.15.122:0/4265505335 wait complete. 2024-08-19T13:14:45.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.746+0000 7f45c2f44640 1 Processor -- start 2024-08-19T13:14:45.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.746+0000 7f45c2f44640 1 -- start start 2024-08-19T13:14:45.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c2f44640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45bc071c20 0x7f45bc06e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c2f44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45bc072d50 0x7f45bc1492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c2f44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45bc1403f0 0x7f45bc13c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c2f44640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc14ada0 con 0x7f45bc1403f0 2024-08-19T13:14:45.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c2f44640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc14ac20 con 0x7f45bc072d50 2024-08-19T13:14:45.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c2f44640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc14af20 con 0x7f45bc071c20 2024-08-19T13:14:45.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c0ce0640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45bc071c20 0x7f45bc06e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45bc072d50 0x7f45bc1492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c0ce0640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45bc071c20 0x7f45bc06e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.122:53456/0 (socket says 172.21.15.122:53456) 2024-08-19T13:14:45.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c14e1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45bc1403f0 0x7f45bc13c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c0ce0640 1 -- 172.21.15.122:0/1906155849 learned_addr learned my addr 172.21.15.122:0/1906155849 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-08-19T13:14:45.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c14e1640 1 -- 172.21.15.122:0/1906155849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45bc071c20 msgr2=0x7f45bc06e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c14e1640 1 --2- 172.21.15.122:0/1906155849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45bc071c20 0x7f45bc06e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c14e1640 1 -- 172.21.15.122:0/1906155849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45bc072d50 msgr2=0x7f45bc1492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c14e1640 1 --2- 172.21.15.122:0/1906155849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45bc072d50 0x7f45bc1492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c14e1640 1 -- 172.21.15.122:0/1906155849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc13d3b0 con 0x7f45bc1403f0 2024-08-19T13:14:45.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.747+0000 7f45c0ce0640 1 --2- 172.21.15.122:0/1906155849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45bc071c20 0x7f45bc06e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:45.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.748+0000 7f45c14e1640 1 --2- 172.21.15.122:0/1906155849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45bc1403f0 0x7f45bc13c000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f45b400c750 tx=0x7f45b400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:45.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.748+0000 7f45b97fa640 1 -- 172.21.15.122:0/1906155849 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4018070 con 0x7f45bc1403f0 2024-08-19T13:14:45.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.748+0000 7f45b97fa640 1 -- 172.21.15.122:0/1906155849 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b4004d60 con 0x7f45bc1403f0 2024-08-19T13:14:45.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.748+0000 7f45c2f44640 1 -- 172.21.15.122:0/1906155849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc13d900 con 0x7f45bc1403f0 2024-08-19T13:14:45.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.748+0000 7f45b97fa640 1 -- 172.21.15.122:0/1906155849 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b400a880 con 0x7f45bc1403f0 2024-08-19T13:14:45.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.748+0000 7f45c2f44640 1 -- 172.21.15.122:0/1906155849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc13c540 con 0x7f45bc1403f0 2024-08-19T13:14:45.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.748+0000 7f45b97fa640 1 -- 172.21.15.122:0/1906155849 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f45b401b070 con 0x7f45bc1403f0 2024-08-19T13:14:45.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.748+0000 7f45b97fa640 1 -- 172.21.15.122:0/1906155849 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f45b40070d0 con 0x7f45bc1403f0 2024-08-19T13:14:45.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.749+0000 7f45c2f44640 1 -- 172.21.15.122:0/1906155849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45bc04fce0 con 0x7f45bc1403f0 2024-08-19T13:14:45.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.751+0000 7f45b97fa640 1 -- 172.21.15.122:0/1906155849 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f45bc04fce0 con 0x7f45bc1403f0 2024-08-19T13:14:45.771 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:45.770+0000 7ffaf1e15040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-19T13:14:45.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.818+0000 7f45c2f44640 1 -- 172.21.15.122:0/1906155849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fab2d2d2-c670-4f56-b099-ca4686395179", "id": 6} v 0) -- 0x7f45bc1408a0 con 0x7f45bc1403f0 2024-08-19T13:14:45.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.820+0000 7f45b97fa640 1 -- 172.21.15.122:0/1906155849 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fab2d2d2-c670-4f56-b099-ca4686395179", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f45b4007670 con 0x7f45bc1403f0 2024-08-19T13:14:45.820 INFO:teuthology.orchestra.run.smithi122.stdout:6 2024-08-19T13:14:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.821+0000 7f45c2f44640 1 -- 172.21.15.122:0/1906155849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45bc1403f0 msgr2=0x7f45bc13c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.821+0000 7f45c2f44640 1 --2- 172.21.15.122:0/1906155849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45bc1403f0 0x7f45bc13c000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f45b400c750 tx=0x7f45b400cc10 comp rx=0 tx=0).stop 2024-08-19T13:14:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.821+0000 7f45c2f44640 1 -- 172.21.15.122:0/1906155849 shutdown_connections 2024-08-19T13:14:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.821+0000 7f45c2f44640 1 --2- 172.21.15.122:0/1906155849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45bc1403f0 0x7f45bc13c000 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.821+0000 7f45c2f44640 1 --2- 172.21.15.122:0/1906155849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45bc072d50 0x7f45bc1492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.821+0000 7f45c2f44640 1 --2- 172.21.15.122:0/1906155849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45bc071c20 0x7f45bc06e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.821+0000 7f45c2f44640 1 -- 172.21.15.122:0/1906155849 >> 172.21.15.122:0/1906155849 conn(0x7f45bc0b6330 msgr2=0x7f45bc0b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.821+0000 7f45c2f44640 1 -- 172.21.15.122:0/1906155849 shutdown_connections 2024-08-19T13:14:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.821+0000 7f45c2f44640 1 -- 172.21.15.122:0/1906155849 wait complete. 2024-08-19T13:14:45.823 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:45.822+0000 7f3838329040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-19T13:14:45.832 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new ee58594c-242f-436e-9c9e-977e57b6785f 7 2024-08-19T13:14:45.906 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:45.906+0000 7f3838329040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-19T13:14:45.933 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:45.933+0000 7ffaf1e15040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-19T13:14:45.953 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:45.952+0000 7f3838329040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-19T13:14:45.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.969+0000 7fc407294640 1 Processor -- start 2024-08-19T13:14:45.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.969+0000 7fc407294640 1 -- start start 2024-08-19T13:14:45.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.969+0000 7fc407294640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc40013fde0 0x7fc400142480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.969+0000 7fc407294640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4000b0b70 0x7fc4001429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.969+0000 7fc407294640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc400070a20 con 0x7fc400067960 2024-08-19T13:14:45.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.969+0000 7fc407294640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc400067d40 con 0x7fc40013fde0 2024-08-19T13:14:45.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.969+0000 7fc407294640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc4000b02d0 con 0x7fc4000b0b70 2024-08-19T13:14:45.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.970+0000 7fc40482f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc40013fde0 0x7fc400142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.970+0000 7fc405030640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fc400067960 0x7fc40013f6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:33370/0 (socket says 172.21.15.122:33370) 2024-08-19T13:14:45.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.970+0000 7fc405030640 1 -- 172.21.15.122:0/3194084082 learned_addr learned my addr 172.21.15.122:0/3194084082 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-08-19T13:14:45.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.970+0000 7fc405831640 1 --2- 172.21.15.122:0/3194084082 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4000b0b70 0x7fc4001429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.970+0000 7fc40482f640 1 -- 172.21.15.122:0/3194084082 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4000b0b70 msgr2=0x7fc4001429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:45.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.970+0000 7fc40482f640 1 --2- 172.21.15.122:0/3194084082 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4000b0b70 0x7fc4001429c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.970+0000 7fc40482f640 1 -- 172.21.15.122:0/3194084082 >> v1:172.21.15.122:6789/0 conn(0x7fc400067960 legacy=0x7fc40013f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.970+0000 7fc40482f640 1 -- 172.21.15.122:0/3194084082 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc400142f00 con 0x7fc40013fde0 2024-08-19T13:14:45.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.971+0000 7fc40482f640 1 --2- 172.21.15.122:0/3194084082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40013fde0 0x7fc400142480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0097a0 tx=0x7fc3fc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3d35b3922cad91 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:45.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.971+0000 7fc3f7fff640 1 -- 172.21.15.122:0/3194084082 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc037070 con 0x7fc40013fde0 2024-08-19T13:14:45.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.971+0000 7fc3f7fff640 1 -- 172.21.15.122:0/3194084082 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3fc02ae50 con 0x7fc40013fde0 2024-08-19T13:14:45.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.971+0000 7fc3f7fff640 1 -- 172.21.15.122:0/3194084082 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc032660 con 0x7fc40013fde0 2024-08-19T13:14:45.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.971+0000 7fc407294640 1 -- 172.21.15.122:0/3194084082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40013fde0 msgr2=0x7fc400142480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.971+0000 7fc407294640 1 --2- 172.21.15.122:0/3194084082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40013fde0 0x7fc400142480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0097a0 tx=0x7fc3fc02a100 comp rx=0 tx=0).stop 2024-08-19T13:14:45.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.971+0000 7fc407294640 1 -- 172.21.15.122:0/3194084082 shutdown_connections 2024-08-19T13:14:45.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.971+0000 7fc407294640 1 --2- 172.21.15.122:0/3194084082 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4000b0b70 0x7fc4001429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.971+0000 7fc407294640 1 --2- 172.21.15.122:0/3194084082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40013fde0 0x7fc400142480 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.971+0000 7fc407294640 1 -- 172.21.15.122:0/3194084082 >> 172.21.15.122:0/3194084082 conn(0x7fc4000b4a20 msgr2=0x7fc4000b4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:45.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.972+0000 7fc407294640 1 -- 172.21.15.122:0/3194084082 shutdown_connections 2024-08-19T13:14:45.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.972+0000 7fc407294640 1 -- 172.21.15.122:0/3194084082 wait complete. 2024-08-19T13:14:45.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.973+0000 7fc407294640 1 Processor -- start 2024-08-19T13:14:45.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.973+0000 7fc407294640 1 -- start start 2024-08-19T13:14:45.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.973+0000 7fc407294640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc400067960 0x7fc400148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.973+0000 7fc407294640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4000b0b70 0x7fc400147a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.973+0000 7fc407294640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc40013fde0 0x7fc40014a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:45.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.973+0000 7fc407294640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc40006f9f0 con 0x7fc40013fde0 2024-08-19T13:14:45.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.973+0000 7fc407294640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc40006f870 con 0x7fc400067960 2024-08-19T13:14:45.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.973+0000 7fc407294640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc40014cd30 con 0x7fc4000b0b70 2024-08-19T13:14:45.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.973+0000 7fc405030640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc400067960 0x7fc400148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.973+0000 7fc405030640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc400067960 0x7fc400148b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42780/0 (socket says 172.21.15.122:42780) 2024-08-19T13:14:45.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.973+0000 7fc405030640 1 -- 172.21.15.122:0/1354677707 learned_addr learned my addr 172.21.15.122:0/1354677707 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-08-19T13:14:45.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.973+0000 7fc40482f640 1 --2- 172.21.15.122:0/1354677707 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4000b0b70 0x7fc400147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.973+0000 7fc405831640 1 --2- 172.21.15.122:0/1354677707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc40013fde0 0x7fc40014a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:45.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.974+0000 7fc405030640 1 -- 172.21.15.122:0/1354677707 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4000b0b70 msgr2=0x7fc400147a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.974+0000 7fc405030640 1 --2- 172.21.15.122:0/1354677707 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4000b0b70 0x7fc400147a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.974+0000 7fc405030640 1 -- 172.21.15.122:0/1354677707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc40013fde0 msgr2=0x7fc40014a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:45.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.974+0000 7fc405030640 1 --2- 172.21.15.122:0/1354677707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc40013fde0 0x7fc40014a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:45.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.974+0000 7fc405030640 1 -- 172.21.15.122:0/1354677707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc400148310 con 0x7fc400067960 2024-08-19T13:14:45.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.974+0000 7fc405030640 1 --2- 172.21.15.122:0/1354677707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc400067960 0x7fc400148b10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000c430 tx=0x7fc3f000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:45.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.974+0000 7fc3f5ffb640 1 -- 172.21.15.122:0/1354677707 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0017070 con 0x7fc400067960 2024-08-19T13:14:45.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.974+0000 7fc3f5ffb640 1 -- 172.21.15.122:0/1354677707 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f0005ce0 con 0x7fc400067960 2024-08-19T13:14:45.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.974+0000 7fc407294640 1 -- 172.21.15.122:0/1354677707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4001485f0 con 0x7fc400067960 2024-08-19T13:14:45.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.974+0000 7fc3f5ffb640 1 -- 172.21.15.122:0/1354677707 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f00072f0 con 0x7fc400067960 2024-08-19T13:14:45.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.974+0000 7fc407294640 1 -- 172.21.15.122:0/1354677707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc400146af0 con 0x7fc400067960 2024-08-19T13:14:45.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.974+0000 7fc3f5ffb640 1 -- 172.21.15.122:0/1354677707 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc3f001a070 con 0x7fc400067960 2024-08-19T13:14:45.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.974+0000 7fc3f5ffb640 1 -- 172.21.15.122:0/1354677707 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fc3f0012d60 con 0x7fc400067960 2024-08-19T13:14:45.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.975+0000 7fc407294640 1 -- 172.21.15.122:0/1354677707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40013f6d0 con 0x7fc400067960 2024-08-19T13:14:45.979 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:45.977+0000 7ffaf1e15040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-19T13:14:45.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:45.977+0000 7fc3f5ffb640 1 -- 172.21.15.122:0/1354677707 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7fc3f0007490 con 0x7fc400067960 2024-08-19T13:14:46.010 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:46.009+0000 7f3838329040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-19T13:14:46.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:46.045+0000 7fc407294640 1 -- 172.21.15.122:0/1354677707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ee58594c-242f-436e-9c9e-977e57b6785f", "id": 7} v 0) -- 0x7fc400070630 con 0x7fc400067960 2024-08-19T13:14:46.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:46.051+0000 7fc3f5ffb640 1 -- 172.21.15.122:0/1354677707 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ee58594c-242f-436e-9c9e-977e57b6785f", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fc3f0007670 con 0x7fc400067960 2024-08-19T13:14:46.052 INFO:teuthology.orchestra.run.smithi122.stdout:7 2024-08-19T13:14:46.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:46.052+0000 7fc407294640 1 -- 172.21.15.122:0/1354677707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc400067960 msgr2=0x7fc400148b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:46.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:46.052+0000 7fc407294640 1 --2- 172.21.15.122:0/1354677707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc400067960 0x7fc400148b10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000c430 tx=0x7fc3f000c8f0 comp rx=0 tx=0).stop 2024-08-19T13:14:46.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:46.052+0000 7fc407294640 1 -- 172.21.15.122:0/1354677707 shutdown_connections 2024-08-19T13:14:46.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:46.052+0000 7fc407294640 1 --2- 172.21.15.122:0/1354677707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc40013fde0 0x7fc40014a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:46.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:46.052+0000 7fc407294640 1 --2- 172.21.15.122:0/1354677707 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4000b0b70 0x7fc400147a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:46.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:46.052+0000 7fc407294640 1 --2- 172.21.15.122:0/1354677707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc400067960 0x7fc400148b10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:46.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:46.052+0000 7fc407294640 1 -- 172.21.15.122:0/1354677707 >> 172.21.15.122:0/1354677707 conn(0x7fc4000b4a20 msgr2=0x7fc4000ae0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:46.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:46.052+0000 7fc407294640 1 -- 172.21.15.122:0/1354677707 shutdown_connections 2024-08-19T13:14:46.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-08-19T13:14:46.052+0000 7fc407294640 1 -- 172.21.15.122:0/1354677707 wait complete. 2024-08-19T13:14:46.063 INFO:tasks.ceph.osd.0:Restarting daemon 2024-08-19T13:14:46.063 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-08-19T13:14:46.065 INFO:tasks.ceph.osd.0:Started 2024-08-19T13:14:46.066 INFO:tasks.ceph.osd.1:Restarting daemon 2024-08-19T13:14:46.066 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-08-19T13:14:46.068 INFO:tasks.ceph.osd.1:Started 2024-08-19T13:14:46.068 INFO:tasks.ceph.osd.2:Restarting daemon 2024-08-19T13:14:46.068 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-08-19T13:14:46.071 INFO:tasks.ceph.osd.2:Started 2024-08-19T13:14:46.071 INFO:tasks.ceph.osd.3:Restarting daemon 2024-08-19T13:14:46.071 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-08-19T13:14:46.073 INFO:tasks.ceph.osd.3:Started 2024-08-19T13:14:46.073 INFO:tasks.ceph.osd.4:Restarting daemon 2024-08-19T13:14:46.074 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-08-19T13:14:46.105 INFO:tasks.ceph.osd.4:Started 2024-08-19T13:14:46.105 INFO:tasks.ceph.osd.5:Restarting daemon 2024-08-19T13:14:46.105 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-08-19T13:14:46.107 INFO:tasks.ceph.osd.5:Started 2024-08-19T13:14:46.107 INFO:tasks.ceph.osd.6:Restarting daemon 2024-08-19T13:14:46.107 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-08-19T13:14:46.109 INFO:tasks.ceph.osd.6:Started 2024-08-19T13:14:46.109 INFO:tasks.ceph.osd.7:Restarting daemon 2024-08-19T13:14:46.110 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-08-19T13:14:46.112 INFO:tasks.ceph.osd.7:Started 2024-08-19T13:14:46.112 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-19T13:14:46.178 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:46.177+0000 7ffaf1e15040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-19T13:14:46.194 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:46.193+0000 7f3838329040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-19T13:14:46.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.228+0000 7f7ef3581640 1 Processor -- start 2024-08-19T13:14:46.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.228+0000 7f7ef3581640 1 -- start start 2024-08-19T13:14:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.229+0000 7f7ef3581640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eec0afff0 0x7f7eec13dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.229+0000 7f7ef3581640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7eec1403f0 0x7f7eec1408a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.229+0000 7f7ef3581640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eec073ae0 con 0x7f7eec071890 2024-08-19T13:14:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.229+0000 7f7ef3581640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f7eec071c70 con 0x7f7eec1403f0 2024-08-19T13:14:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.229+0000 7f7ef3581640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7eec0729f0 con 0x7f7eec0afff0 2024-08-19T13:14:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.229+0000 7f7ef131d640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f7eec071890 0x7f7eec13d490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:41868/0 (socket says 172.21.15.103:41868) 2024-08-19T13:14:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.229+0000 7f7ef0b1c640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eec0afff0 0x7f7eec13dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.229+0000 7f7ef131d640 1 -- 172.21.15.103:0/1053168591 learned_addr learned my addr 172.21.15.103:0/1053168591 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:14:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.229+0000 7f7ef1b1e640 1 --2- 172.21.15.103:0/1053168591 >> v2:172.21.15.122:3300/0 conn(0x7f7eec1403f0 0x7f7eec1408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.229+0000 7f7ee3fff640 1 -- 172.21.15.103:0/1053168591 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 970065792 0 0) 0x7f7eec073ae0 con 0x7f7eec071890 2024-08-19T13:14:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.229+0000 7f7ef0b1c640 1 -- 172.21.15.103:0/1053168591 >> v2:172.21.15.122:3300/0 conn(0x7f7eec1403f0 msgr2=0x7f7eec1408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.229+0000 7f7ef0b1c640 1 --2- 172.21.15.103:0/1053168591 >> v2:172.21.15.122:3300/0 conn(0x7f7eec1403f0 0x7f7eec1408a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.229+0000 7f7ef0b1c640 1 -- 172.21.15.103:0/1053168591 >> v1:172.21.15.103:6789/0 conn(0x7f7eec071890 legacy=0x7f7eec13d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.229+0000 7f7ef0b1c640 1 -- 172.21.15.103:0/1053168591 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eec0b0520 con 0x7f7eec0afff0 2024-08-19T13:14:46.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.230+0000 7f7ef0b1c640 1 --2- 172.21.15.103:0/1053168591 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eec0afff0 0x7f7eec13dba0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80097a0 tx=0x7f7ee802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=523a96a4400713ff server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:46.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.230+0000 7f7ee3fff640 1 -- 172.21.15.103:0/1053168591 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8037070 con 0x7f7eec0afff0 2024-08-19T13:14:46.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.230+0000 7f7ee3fff640 1 -- 172.21.15.103:0/1053168591 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee802adb0 con 0x7f7eec0afff0 2024-08-19T13:14:46.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.230+0000 7f7ee3fff640 1 -- 172.21.15.103:0/1053168591 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8032660 con 0x7f7eec0afff0 2024-08-19T13:14:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.231+0000 7f7ef3581640 1 -- 172.21.15.103:0/1053168591 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eec0afff0 msgr2=0x7f7eec13dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.231+0000 7f7ef3581640 1 --2- 172.21.15.103:0/1053168591 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eec0afff0 0x7f7eec13dba0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80097a0 tx=0x7f7ee802a060 comp rx=0 tx=0).stop 2024-08-19T13:14:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.231+0000 7f7ef3581640 1 -- 172.21.15.103:0/1053168591 shutdown_connections 2024-08-19T13:14:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.231+0000 7f7ef3581640 1 --2- 172.21.15.103:0/1053168591 >> v2:172.21.15.122:3300/0 conn(0x7f7eec1403f0 0x7f7eec1408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.231+0000 7f7ef3581640 1 --2- 172.21.15.103:0/1053168591 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eec0afff0 0x7f7eec13dba0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.231+0000 7f7ef3581640 1 -- 172.21.15.103:0/1053168591 >> 172.21.15.103:0/1053168591 conn(0x7f7eec0b5fb0 msgr2=0x7f7eec0b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.231+0000 7f7ef3581640 1 -- 172.21.15.103:0/1053168591 shutdown_connections 2024-08-19T13:14:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.231+0000 7f7ef3581640 1 -- 172.21.15.103:0/1053168591 wait complete. 2024-08-19T13:14:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.232+0000 7f7ef3581640 1 Processor -- start 2024-08-19T13:14:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.232+0000 7f7ef3581640 1 -- start start 2024-08-19T13:14:46.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.232+0000 7f7ef3581640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eec071890 0x7f7eec148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:46.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.232+0000 7f7ef3581640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eec0afff0 0x7f7eec1d5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:46.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.232+0000 7f7ef3581640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec1403f0 0x7f7eec1d60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:46.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.232+0000 7f7ef3581640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec14ab90 con 0x7f7eec0afff0 2024-08-19T13:14:46.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.232+0000 7f7ef3581640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec140ff0 con 0x7f7eec1403f0 2024-08-19T13:14:46.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef3581640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7eec14ad10 con 0x7f7eec071890 2024-08-19T13:14:46.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef131d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eec071890 0x7f7eec148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:46.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef0b1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eec0afff0 0x7f7eec1d5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:46.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef131d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eec071890 0x7f7eec148160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:46106/0 (socket says 172.21.15.103:46106) 2024-08-19T13:14:46.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef131d640 1 -- 172.21.15.103:0/3068033376 learned_addr learned my addr 172.21.15.103:0/3068033376 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:46.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef1b1e640 1 --2- 172.21.15.103:0/3068033376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec1403f0 0x7f7eec1d60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:46.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef0b1c640 1 -- 172.21.15.103:0/3068033376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eec071890 msgr2=0x7f7eec148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:46.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef0b1c640 1 --2- 172.21.15.103:0/3068033376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eec071890 0x7f7eec148160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:46.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef0b1c640 1 -- 172.21.15.103:0/3068033376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec1403f0 msgr2=0x7f7eec1d60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:46.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef0b1c640 1 --2- 172.21.15.103:0/3068033376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec1403f0 0x7f7eec1d60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:46.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef0b1c640 1 -- 172.21.15.103:0/3068033376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eec149a00 con 0x7f7eec0afff0 2024-08-19T13:14:46.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef1b1e640 1 --2- 172.21.15.103:0/3068033376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec1403f0 0x7f7eec1d60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:46.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef131d640 1 --2- 172.21.15.103:0/3068033376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eec071890 0x7f7eec148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:46.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef0b1c640 1 --2- 172.21.15.103:0/3068033376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eec0afff0 0x7f7eec1d5b70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8031000 tx=0x7f7ee80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:46.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ee1ffb640 1 -- 172.21.15.103:0/3068033376 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8041070 con 0x7f7eec0afff0 2024-08-19T13:14:46.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ee1ffb640 1 -- 172.21.15.103:0/3068033376 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee8032e20 con 0x7f7eec0afff0 2024-08-19T13:14:46.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ef3581640 1 -- 172.21.15.103:0/3068033376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eec148b00 con 0x7f7eec0afff0 2024-08-19T13:14:46.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.233+0000 7f7ee1ffb640 1 -- 172.21.15.103:0/3068033376 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8037040 con 0x7f7eec0afff0 2024-08-19T13:14:46.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.234+0000 7f7ef3581640 1 -- 172.21.15.103:0/3068033376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eec144f80 con 0x7f7eec0afff0 2024-08-19T13:14:46.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.234+0000 7f7ee1ffb640 1 -- 172.21.15.103:0/3068033376 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7ee803a070 con 0x7f7eec0afff0 2024-08-19T13:14:46.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.234+0000 7f7ee1ffb640 1 -- 172.21.15.103:0/3068033376 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7ee803dac0 con 0x7f7eec0afff0 2024-08-19T13:14:46.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.234+0000 7f7ef3581640 1 -- 172.21.15.103:0/3068033376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eec06f0a0 con 0x7f7eec0afff0 2024-08-19T13:14:46.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.236+0000 7f7ee1ffb640 1 -- 172.21.15.103:0/3068033376 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f7eec06f0a0 con 0x7f7eec0afff0 2024-08-19T13:14:46.243 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:46.242+0000 7f3838329040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-19T13:14:46.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.294+0000 7f7ef3581640 1 -- 172.21.15.103:0/3068033376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7eec1408a0 con 0x7f7eec0afff0 2024-08-19T13:14:46.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.295+0000 7f7ee1ffb640 1 -- 172.21.15.103:0/3068033376 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7ee803dd10 con 0x7f7eec0afff0 2024-08-19T13:14:46.296 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:46.296 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":10,"fsid":"9a6e456b-d04b-48cb-846f-0fbd1ee79f13","created":"2024-08-19T13:14:43.897750+0000","modified":"2024-08-19T13:14:46.047417+0000","last_up_change":"0.000000","last_in_change":"2024-08-19T13:14:46.047417+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"01d9bd23-e3e8-4905-af40-142b43a27628","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d562d5f9-b0ac-4f9c-856d-cdcf1be3da18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"adc47b95-8fbe-48c0-bb44-f78ba8ed6d29","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a2e730c7-32c8-4387-bdee-e00a70e7613a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"45b5502e-ce4c-411c-b7a2-1742483a432f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"056397bb-673f-461f-87c8-e44390d7ba37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fab2d2d2-c670-4f56-b099-ca4686395179","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ee58594c-242f-436e-9c9e-977e57b6785f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T13:14:46.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.296+0000 7f7ef3581640 1 -- 172.21.15.103:0/3068033376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eec0afff0 msgr2=0x7f7eec1d5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:46.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.296+0000 7f7ef3581640 1 --2- 172.21.15.103:0/3068033376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eec0afff0 0x7f7eec1d5b70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8031000 tx=0x7f7ee80046c0 comp rx=0 tx=0).stop 2024-08-19T13:14:46.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.296+0000 7f7ef3581640 1 -- 172.21.15.103:0/3068033376 shutdown_connections 2024-08-19T13:14:46.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.296+0000 7f7ef3581640 1 --2- 172.21.15.103:0/3068033376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec1403f0 0x7f7eec1d60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:46.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.296+0000 7f7ef3581640 1 --2- 172.21.15.103:0/3068033376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eec0afff0 0x7f7eec1d5b70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:46.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.296+0000 7f7ef3581640 1 --2- 172.21.15.103:0/3068033376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eec071890 0x7f7eec148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:46.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.296+0000 7f7ef3581640 1 -- 172.21.15.103:0/3068033376 >> 172.21.15.103:0/3068033376 conn(0x7f7eec0b5fb0 msgr2=0x7f7eec13b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:46.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.296+0000 7f7ef3581640 1 -- 172.21.15.103:0/3068033376 shutdown_connections 2024-08-19T13:14:46.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:46.296+0000 7f7ef3581640 1 -- 172.21.15.103:0/3068033376 wait complete. 2024-08-19T13:14:46.307 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-08-19T13:14:46.307 INFO:tasks.ceph:Waiting for OSDs to come up 2024-08-19T13:14:46.406 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:46.406+0000 7f3838329040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-19T13:14:46.473 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:46.473+0000 7ffaf1e15040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-19T13:14:46.672 INFO:tasks.ceph.osd.2.smithi103.stderr:2024-08-19T13:14:46.671+0000 7f74e6c2b640 -1 Falling back to public interface 2024-08-19T13:14:46.675 INFO:tasks.ceph.osd.0.smithi103.stderr:2024-08-19T13:14:46.675+0000 7fd2d892c640 -1 Falling back to public interface 2024-08-19T13:14:46.681 INFO:tasks.ceph.osd.3.smithi103.stderr:2024-08-19T13:14:46.680+0000 7f8ac4ddc640 -1 Falling back to public interface 2024-08-19T13:14:46.685 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:46.685+0000 7f3838329040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-19T13:14:46.696 INFO:tasks.ceph.osd.1.smithi103.stderr:2024-08-19T13:14:46.695+0000 7fe604411640 -1 Falling back to public interface 2024-08-19T13:14:46.701 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-08-19T13:14:46.700+0000 7f5a3caf0640 -1 Falling back to public interface 2024-08-19T13:14:46.707 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-08-19T13:14:46.706+0000 7fe994940640 -1 Falling back to public interface 2024-08-19T13:14:46.725 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-08-19T13:14:46.724+0000 7f73935d6640 -1 Falling back to public interface 2024-08-19T13:14:46.728 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-08-19T13:14:46.727+0000 7f02aa388640 -1 Falling back to public interface 2024-08-19T13:14:46.830 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:46.830+0000 7ffaf1e15040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-19T13:14:46.871 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:46.870+0000 7ffaf1e15040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-19T13:14:46.915 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:46.915+0000 7ffaf1e15040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-19T13:14:47.045 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:47.044+0000 7f3838329040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-19T13:14:47.086 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:47.085+0000 7f3838329040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-19T13:14:47.112 DEBUG:teuthology.orchestra.run.smithi103:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-19T13:14:47.128 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-08-19T13:14:47.128+0000 7f3838329040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-19T13:14:47.203 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.202+0000 7fb2744bc640 1 Processor -- start 2024-08-19T13:14:47.203 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.203+0000 7fb2744bc640 1 -- start start 2024-08-19T13:14:47.205 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.203+0000 7fb2744bc640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb26c06bbb0 0x7fb26c0b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:47.205 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.203+0000 7fb2744bc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb26c0af3f0 0x7fb26c13d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:47.205 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.203+0000 7fb2744bc640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb26c0b0ab0 con 0x7fb26c073910 2024-08-19T13:14:47.205 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.203+0000 7fb2744bc640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fb26c06bf90 con 0x7fb26c0af3f0 2024-08-19T13:14:47.205 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.203+0000 7fb2744bc640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb26c0b0050 con 0x7fb26c06bbb0 2024-08-19T13:14:47.205 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.203+0000 7fb272258640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb26c06bbb0 0x7fb26c0b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:47.206 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.203+0000 7fb271a57640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fb26c073910 0x7fb26c143810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:41884/0 (socket says 172.21.15.103:41884) 2024-08-19T13:14:47.206 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.203+0000 7fb271a57640 1 -- 172.21.15.103:0/3648966668 learned_addr learned my addr 172.21.15.103:0/3648966668 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:14:47.206 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.203+0000 7fb272258640 1 --2- 172.21.15.103:0/3648966668 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb26c06bbb0 0x7fb26c0b0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:46118/0 (socket says 172.21.15.103:46118) 2024-08-19T13:14:47.206 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.204+0000 7fb272a59640 1 --2- 172.21.15.103:0/3648966668 >> v2:172.21.15.122:3300/0 conn(0x7fb26c0af3f0 0x7fb26c13d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:47.206 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.204+0000 7fb272258640 1 -- 172.21.15.103:0/3648966668 >> v2:172.21.15.122:3300/0 conn(0x7fb26c0af3f0 msgr2=0x7fb26c13d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:47.206 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.204+0000 7fb272258640 1 --2- 172.21.15.103:0/3648966668 >> v2:172.21.15.122:3300/0 conn(0x7fb26c0af3f0 0x7fb26c13d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:47.206 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.204+0000 7fb272258640 1 -- 172.21.15.103:0/3648966668 >> v1:172.21.15.103:6789/0 conn(0x7fb26c073910 legacy=0x7fb26c143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:47.206 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.204+0000 7fb271256640 1 -- 172.21.15.103:0/3648966668 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1715225110 0 0) 0x7fb26c0b0ab0 con 0x7fb26c073910 2024-08-19T13:14:47.206 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.204+0000 7fb272258640 1 -- 172.21.15.103:0/3648966668 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26c0afc70 con 0x7fb26c06bbb0 2024-08-19T13:14:47.206 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.204+0000 7fb272258640 1 --2- 172.21.15.103:0/3648966668 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb26c06bbb0 0x7fb26c0b0570 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0097d0 tx=0x7fb25c02a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81ae1c0b14b00cbe server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:47.206 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.205+0000 7fb271256640 1 -- 172.21.15.103:0/3648966668 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c037070 con 0x7fb26c06bbb0 2024-08-19T13:14:47.207 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.205+0000 7fb271256640 1 -- 172.21.15.103:0/3648966668 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb25c02ae40 con 0x7fb26c06bbb0 2024-08-19T13:14:47.207 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.205+0000 7fb271256640 1 -- 172.21.15.103:0/3648966668 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c032660 con 0x7fb26c06bbb0 2024-08-19T13:14:47.210 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.205+0000 7fb2744bc640 1 -- 172.21.15.103:0/3648966668 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb26c06bbb0 msgr2=0x7fb26c0b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:47.210 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.205+0000 7fb2744bc640 1 --2- 172.21.15.103:0/3648966668 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb26c06bbb0 0x7fb26c0b0570 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0097d0 tx=0x7fb25c02a130 comp rx=0 tx=0).stop 2024-08-19T13:14:47.210 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.205+0000 7fb2744bc640 1 -- 172.21.15.103:0/3648966668 shutdown_connections 2024-08-19T13:14:47.210 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.205+0000 7fb2744bc640 1 --2- 172.21.15.103:0/3648966668 >> v2:172.21.15.122:3300/0 conn(0x7fb26c0af3f0 0x7fb26c13d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:47.210 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.205+0000 7fb2744bc640 1 --2- 172.21.15.103:0/3648966668 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb26c06bbb0 0x7fb26c0b0570 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:47.210 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.205+0000 7fb2744bc640 1 -- 172.21.15.103:0/3648966668 >> 172.21.15.103:0/3648966668 conn(0x7fb26c0b4fd0 msgr2=0x7fb26c0b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:47.210 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.205+0000 7fb2744bc640 1 -- 172.21.15.103:0/3648966668 shutdown_connections 2024-08-19T13:14:47.210 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.205+0000 7fb2744bc640 1 -- 172.21.15.103:0/3648966668 wait complete. 2024-08-19T13:14:47.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.206+0000 7fb2744bc640 1 Processor -- start 2024-08-19T13:14:47.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.206+0000 7fb2744bc640 1 -- start start 2024-08-19T13:14:47.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb2744bc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb26c06bbb0 0x7fb26c1d8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:47.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb2744bc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb26c073910 0x7fb26c1d93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:47.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb2744bc640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb26c0af3f0 0x7fb26c147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:47.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb2744bc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c13dd90 con 0x7fb26c073910 2024-08-19T13:14:47.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb2744bc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c13dc10 con 0x7fb26c06bbb0 2024-08-19T13:14:47.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb2744bc640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb26c13df10 con 0x7fb26c0af3f0 2024-08-19T13:14:47.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb271a57640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb26c073910 0x7fb26c1d93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:47.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb272a59640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb26c0af3f0 0x7fb26c147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:47.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb272a59640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb26c0af3f0 0x7fb26c147b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:46122/0 (socket says 172.21.15.103:46122) 2024-08-19T13:14:47.212 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb272258640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb26c06bbb0 0x7fb26c1d8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:47.212 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb272a59640 1 -- 172.21.15.103:0/3262021241 learned_addr learned my addr 172.21.15.103:0/3262021241 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:47.212 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb271a57640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb26c073910 0x7fb26c1d93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52532/0 (socket says 172.21.15.103:52532) 2024-08-19T13:14:47.212 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb272a59640 1 -- 172.21.15.103:0/3262021241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb26c06bbb0 msgr2=0x7fb26c1d8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:47.212 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb272a59640 1 --2- 172.21.15.103:0/3262021241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb26c06bbb0 0x7fb26c1d8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:47.212 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb272a59640 1 -- 172.21.15.103:0/3262021241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb26c073910 msgr2=0x7fb26c1d93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:47.212 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb272a59640 1 --2- 172.21.15.103:0/3262021241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb26c073910 0x7fb26c1d93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:47.212 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb272a59640 1 -- 172.21.15.103:0/3262021241 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26c149520 con 0x7fb26c0af3f0 2024-08-19T13:14:47.212 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb271a57640 1 --2- 172.21.15.103:0/3262021241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb26c073910 0x7fb26c1d93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:47.212 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.207+0000 7fb272a59640 1 --2- 172.21.15.103:0/3262021241 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb26c0af3f0 0x7fb26c147b60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb26400e7f0 tx=0x7fb26400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:47.212 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.208+0000 7fb26affd640 1 -- 172.21.15.103:0/3262021241 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb264017070 con 0x7fb26c0af3f0 2024-08-19T13:14:47.212 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.208+0000 7fb26affd640 1 -- 172.21.15.103:0/3262021241 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb26400c730 con 0x7fb26c0af3f0 2024-08-19T13:14:47.213 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.208+0000 7fb26affd640 1 -- 172.21.15.103:0/3262021241 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26400cb20 con 0x7fb26c0af3f0 2024-08-19T13:14:47.213 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.208+0000 7fb2744bc640 1 -- 172.21.15.103:0/3262021241 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26c1d8980 con 0x7fb26c0af3f0 2024-08-19T13:14:47.213 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.208+0000 7fb2744bc640 1 -- 172.21.15.103:0/3262021241 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb26c1480a0 con 0x7fb26c0af3f0 2024-08-19T13:14:47.213 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.208+0000 7fb26affd640 1 -- 172.21.15.103:0/3262021241 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb26401a070 con 0x7fb26c0af3f0 2024-08-19T13:14:47.213 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.208+0000 7fb26affd640 1 -- 172.21.15.103:0/3262021241 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fb26401cb20 con 0x7fb26c0af3f0 2024-08-19T13:14:47.213 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.209+0000 7fb2744bc640 1 -- 172.21.15.103:0/3262021241 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb26c13d4c0 con 0x7fb26c0af3f0 2024-08-19T13:14:47.213 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.211+0000 7fb26affd640 1 -- 172.21.15.103:0/3262021241 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7fb26400cd60 con 0x7fb26c0af3f0 2024-08-19T13:14:47.289 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.289+0000 7fb2744bc640 1 -- 172.21.15.103:0/3262021241 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb26c075ac0 con 0x7fb26c0af3f0 2024-08-19T13:14:47.290 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.289+0000 7fb26affd640 1 -- 172.21.15.103:0/3262021241 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fb26401c450 con 0x7fb26c0af3f0 2024-08-19T13:14:47.290 INFO:teuthology.misc.health.smithi103.stdout: 2024-08-19T13:14:47.290 INFO:teuthology.misc.health.smithi103.stdout:{"epoch":10,"fsid":"9a6e456b-d04b-48cb-846f-0fbd1ee79f13","created":"2024-08-19T13:14:43.897750+0000","modified":"2024-08-19T13:14:46.047417+0000","last_up_change":"0.000000","last_in_change":"2024-08-19T13:14:46.047417+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"01d9bd23-e3e8-4905-af40-142b43a27628","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d562d5f9-b0ac-4f9c-856d-cdcf1be3da18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"adc47b95-8fbe-48c0-bb44-f78ba8ed6d29","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a2e730c7-32c8-4387-bdee-e00a70e7613a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"45b5502e-ce4c-411c-b7a2-1742483a432f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"056397bb-673f-461f-87c8-e44390d7ba37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fab2d2d2-c670-4f56-b099-ca4686395179","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ee58594c-242f-436e-9c9e-977e57b6785f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T13:14:47.291 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.290+0000 7fb2744bc640 1 -- 172.21.15.103:0/3262021241 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb26c0af3f0 msgr2=0x7fb26c147b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:47.291 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.290+0000 7fb2744bc640 1 --2- 172.21.15.103:0/3262021241 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb26c0af3f0 0x7fb26c147b60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb26400e7f0 tx=0x7fb26400ecb0 comp rx=0 tx=0).stop 2024-08-19T13:14:47.291 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.290+0000 7fb2744bc640 1 -- 172.21.15.103:0/3262021241 shutdown_connections 2024-08-19T13:14:47.292 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.290+0000 7fb2744bc640 1 --2- 172.21.15.103:0/3262021241 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb26c0af3f0 0x7fb26c147b60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:47.292 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.290+0000 7fb2744bc640 1 --2- 172.21.15.103:0/3262021241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb26c073910 0x7fb26c1d93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:47.292 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.290+0000 7fb2744bc640 1 --2- 172.21.15.103:0/3262021241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb26c06bbb0 0x7fb26c1d8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:47.292 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.290+0000 7fb2744bc640 1 -- 172.21.15.103:0/3262021241 >> 172.21.15.103:0/3262021241 conn(0x7fb26c0b4fd0 msgr2=0x7fb26c13c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:47.292 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.290+0000 7fb2744bc640 1 -- 172.21.15.103:0/3262021241 shutdown_connections 2024-08-19T13:14:47.292 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:47.290+0000 7fb2744bc640 1 -- 172.21.15.103:0/3262021241 wait complete. 2024-08-19T13:14:47.300 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-08-19T13:14:48.999 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:48.999+0000 7ffa91ac0640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-08-19T13:14:49.001 INFO:tasks.ceph.mgr.y.smithi103.stderr:2024-08-19T13:14:49.001+0000 7ffa91ac0640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-08-19T13:14:50.162 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-08-19T13:14:50.161+0000 7f5a3caf0640 -1 osd.4 0 log_to_monitors true 2024-08-19T13:14:50.169 INFO:tasks.ceph.osd.3.smithi103.stderr:2024-08-19T13:14:50.168+0000 7f8ac4ddc640 -1 osd.3 0 log_to_monitors true 2024-08-19T13:14:50.206 INFO:tasks.ceph.osd.2.smithi103.stderr:2024-08-19T13:14:50.206+0000 7f74e6c2b640 -1 osd.2 0 log_to_monitors true 2024-08-19T13:14:50.207 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-08-19T13:14:50.206+0000 7fe994940640 -1 osd.6 0 log_to_monitors true 2024-08-19T13:14:50.219 INFO:tasks.ceph.osd.1.smithi103.stderr:2024-08-19T13:14:50.218+0000 7fe604411640 -1 osd.1 0 log_to_monitors true 2024-08-19T13:14:50.225 INFO:tasks.ceph.osd.0.smithi103.stderr:2024-08-19T13:14:50.225+0000 7fd2d892c640 -1 osd.0 0 log_to_monitors true 2024-08-19T13:14:50.233 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-08-19T13:14:50.233+0000 7f73935d6640 -1 osd.5 0 log_to_monitors true 2024-08-19T13:14:50.250 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-08-19T13:14:50.249+0000 7f02aa388640 -1 osd.7 0 log_to_monitors true 2024-08-19T13:14:52.013 INFO:tasks.ceph.osd.1.smithi103.stderr:2024-08-19T13:14:52.012+0000 7fe5ffd24640 -1 osd.1 0 waiting for initial osdmap 2024-08-19T13:14:52.015 INFO:tasks.ceph.osd.3.smithi103.stderr:2024-08-19T13:14:52.015+0000 7f8ac0d72640 -1 osd.3 0 waiting for initial osdmap 2024-08-19T13:14:52.017 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-08-19T13:14:52.016+0000 7f02a631e640 -1 osd.7 0 waiting for initial osdmap 2024-08-19T13:14:52.020 INFO:tasks.ceph.osd.0.smithi103.stderr:2024-08-19T13:14:52.020+0000 7fd2d48c2640 -1 osd.0 0 waiting for initial osdmap 2024-08-19T13:14:52.026 INFO:tasks.ceph.osd.0.smithi103.stderr:2024-08-19T13:14:52.026+0000 7fd2d06ea640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T13:14:52.027 INFO:tasks.ceph.osd.1.smithi103.stderr:2024-08-19T13:14:52.027+0000 7fe5fbb4c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T13:14:52.030 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-08-19T13:14:52.030+0000 7f02a2146640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T13:14:52.031 INFO:tasks.ceph.osd.3.smithi103.stderr:2024-08-19T13:14:52.031+0000 7f8abcb9a640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T13:14:52.602 INFO:tasks.ceph.osd.2.smithi103.stderr:2024-08-19T13:14:52.602+0000 7f74e2bc1640 -1 osd.2 0 waiting for initial osdmap 2024-08-19T13:14:52.616 INFO:tasks.ceph.osd.2.smithi103.stderr:2024-08-19T13:14:52.616+0000 7f74da9e1640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T13:14:52.845 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-08-19T13:14:52.844+0000 7fe9908d6640 -1 osd.6 0 waiting for initial osdmap 2024-08-19T13:14:52.854 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-08-19T13:14:52.854+0000 7f738f56c640 -1 osd.5 0 waiting for initial osdmap 2024-08-19T13:14:52.859 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-08-19T13:14:52.859+0000 7fe9886f6640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T13:14:52.864 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-08-19T13:14:52.864+0000 7f738738c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T13:14:52.883 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-08-19T13:14:52.882+0000 7f5a38a86640 -1 osd.4 0 waiting for initial osdmap 2024-08-19T13:14:52.892 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-08-19T13:14:52.892+0000 7f5a308a6640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T13:14:54.108 DEBUG:teuthology.orchestra.run.smithi103:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-19T13:14:54.207 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.206+0000 7f0434813640 1 Processor -- start 2024-08-19T13:14:54.207 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.206+0000 7f0434813640 1 -- start start 2024-08-19T13:14:54.208 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.207+0000 7f0434813640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f043006cbc0 0x7f0430142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:54.208 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.207+0000 7f0434813640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f0430072610 0x7f04300ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:54.208 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.207+0000 7f0434813640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f043006f880 con 0x7f0430072a30 2024-08-19T13:14:54.208 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.207+0000 7f0434813640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f043006cfa0 con 0x7f0430072610 2024-08-19T13:14:54.209 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.207+0000 7f0434813640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f043006fb60 con 0x7f043006cbc0 2024-08-19T13:14:54.209 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.207+0000 7f042e59c640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f043006cbc0 0x7f0430142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:54.209 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.207+0000 7f0427fff640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f0430072610 0x7f04300ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:54.209 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.207+0000 7f042e59c640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f043006cbc0 0x7f0430142980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:46232/0 (socket says 172.21.15.103:46232) 2024-08-19T13:14:54.209 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.207+0000 7f042e59c640 1 -- 172.21.15.103:0/1028527515 learned_addr learned my addr 172.21.15.103:0/1028527515 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:54.209 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.208+0000 7f0427fff640 1 -- 172.21.15.103:0/1028527515 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f043006cbc0 msgr2=0x7f0430142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:54.209 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.208+0000 7f0427fff640 1 --2- 172.21.15.103:0/1028527515 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f043006cbc0 0x7f0430142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:54.209 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.208+0000 7f0427fff640 1 -- 172.21.15.103:0/1028527515 >> v1:172.21.15.122:6789/0 conn(0x7f0430072a30 legacy=0x7f04300addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:54.209 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.208+0000 7f0427fff640 1 -- 172.21.15.103:0/1028527515 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0430142f80 con 0x7f0430072610 2024-08-19T13:14:54.209 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.208+0000 7f042e59c640 1 --2- 172.21.15.103:0/1028527515 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f043006cbc0 0x7f0430142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:54.210 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.208+0000 7f0427fff640 1 --2- 172.21.15.103:0/1028527515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0430072610 0x7f04300ad890 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f04180097a0 tx=0x7f041802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b05b0a8f9365a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:54.210 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.208+0000 7f042dd9b640 1 -- 172.21.15.103:0/1028527515 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418037070 con 0x7f0430072610 2024-08-19T13:14:54.210 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.208+0000 7f042dd9b640 1 -- 172.21.15.103:0/1028527515 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f041802ae50 con 0x7f0430072610 2024-08-19T13:14:54.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.209+0000 7f042dd9b640 1 -- 172.21.15.103:0/1028527515 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418032660 con 0x7f0430072610 2024-08-19T13:14:54.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.209+0000 7f0434813640 1 -- 172.21.15.103:0/1028527515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0430072610 msgr2=0x7f04300ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:54.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.209+0000 7f0434813640 1 --2- 172.21.15.103:0/1028527515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0430072610 0x7f04300ad890 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f04180097a0 tx=0x7f041802a100 comp rx=0 tx=0).stop 2024-08-19T13:14:54.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.209+0000 7f0434813640 1 -- 172.21.15.103:0/1028527515 shutdown_connections 2024-08-19T13:14:54.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.209+0000 7f0434813640 1 --2- 172.21.15.103:0/1028527515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0430072610 0x7f04300ad890 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:54.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.209+0000 7f0434813640 1 --2- 172.21.15.103:0/1028527515 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f043006cbc0 0x7f0430142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:54.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.209+0000 7f0434813640 1 -- 172.21.15.103:0/1028527515 >> 172.21.15.103:0/1028527515 conn(0x7f04300b1340 msgr2=0x7f04300b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:54.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.209+0000 7f0434813640 1 -- 172.21.15.103:0/1028527515 shutdown_connections 2024-08-19T13:14:54.211 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.209+0000 7f0434813640 1 -- 172.21.15.103:0/1028527515 wait complete. 2024-08-19T13:14:54.213 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.210+0000 7f0434813640 1 Processor -- start 2024-08-19T13:14:54.213 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.210+0000 7f0434813640 1 -- start start 2024-08-19T13:14:54.213 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f0434813640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f043006cbc0 0x7f0430147450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:54.213 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f0434813640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0430072610 0x7f0430146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:54.214 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f0434813640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0430072a30 0x7f0430143840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:54.214 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f0434813640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f043014cf50 con 0x7f0430072a30 2024-08-19T13:14:54.214 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f0434813640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f043014cdd0 con 0x7f043006cbc0 2024-08-19T13:14:54.214 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f0434813640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f043014d0d0 con 0x7f0430072610 2024-08-19T13:14:54.214 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f0427fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0430072610 0x7f0430146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:54.214 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f042ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0430072a30 0x7f0430143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:54.214 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f0427fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0430072610 0x7f0430146360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:46242/0 (socket says 172.21.15.103:46242) 2024-08-19T13:14:54.214 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f0427fff640 1 -- 172.21.15.103:0/147735325 learned_addr learned my addr 172.21.15.103:0/147735325 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:54.214 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f042ed9d640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0430072a30 0x7f0430143840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52674/0 (socket says 172.21.15.103:52674) 2024-08-19T13:14:54.214 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f042e59c640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f043006cbc0 0x7f0430147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:54.214 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f0427fff640 1 -- 172.21.15.103:0/147735325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f043006cbc0 msgr2=0x7f0430147450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:54.214 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f0427fff640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f043006cbc0 0x7f0430147450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:54.215 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f0427fff640 1 -- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0430072a30 msgr2=0x7f0430143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:54.215 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f0427fff640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0430072a30 0x7f0430143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:54.215 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f0427fff640 1 -- 172.21.15.103:0/147735325 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0430148450 con 0x7f0430072610 2024-08-19T13:14:54.215 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.211+0000 7f042ed9d640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0430072a30 0x7f0430143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:54.215 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.212+0000 7f0427fff640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0430072610 0x7f0430146360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f04180098c0 tx=0x7f04180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:54.215 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.212+0000 7f0426ffd640 1 -- 172.21.15.103:0/147735325 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418041030 con 0x7f0430072610 2024-08-19T13:14:54.215 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.212+0000 7f0426ffd640 1 -- 172.21.15.103:0/147735325 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f041802ac50 con 0x7f0430072610 2024-08-19T13:14:54.215 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.212+0000 7f0434813640 1 -- 172.21.15.103:0/147735325 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0430148730 con 0x7f0430072610 2024-08-19T13:14:54.215 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.212+0000 7f0426ffd640 1 -- 172.21.15.103:0/147735325 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418037040 con 0x7f0430072610 2024-08-19T13:14:54.215 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.212+0000 7f0434813640 1 -- 172.21.15.103:0/147735325 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0430146b90 con 0x7f0430072610 2024-08-19T13:14:54.215 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.213+0000 7f0434813640 1 -- 172.21.15.103:0/147735325 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0430142980 con 0x7f0430072610 2024-08-19T13:14:54.217 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.213+0000 7f0426ffd640 1 -- 172.21.15.103:0/147735325 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f0418044070 con 0x7f0430072610 2024-08-19T13:14:54.217 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.214+0000 7f0426ffd640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f041006bda0 0x7f041006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:54.217 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.214+0000 7f0426ffd640 1 -- 172.21.15.103:0/147735325 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(14..14 src has 1..14) ==== 4846+0+0 (secure 0 0 0) 0x7f04180aaf10 con 0x7f0430072610 2024-08-19T13:14:54.217 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.214+0000 7f042e59c640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f041006bda0 0x7f041006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:54.217 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.214+0000 7f042e59c640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f041006bda0 0x7f041006e260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0420009b70 tx=0x7f0420008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:54.219 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.218+0000 7f0426ffd640 1 -- 172.21.15.103:0/147735325 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f0418079e20 con 0x7f0430072610 2024-08-19T13:14:54.385 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.384+0000 7f0434813640 1 -- 172.21.15.103:0/147735325 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0430148ac0 con 0x7f0430072610 2024-08-19T13:14:54.386 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.385+0000 7f0426ffd640 1 -- 172.21.15.103:0/147735325 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+12912 (secure 0 0 0) 0x7f041802ae20 con 0x7f0430072610 2024-08-19T13:14:54.386 INFO:teuthology.misc.health.smithi103.stdout: 2024-08-19T13:14:54.386 INFO:teuthology.misc.health.smithi103.stdout:{"epoch":14,"fsid":"9a6e456b-d04b-48cb-846f-0fbd1ee79f13","created":"2024-08-19T13:14:43.897750+0000","modified":"2024-08-19T13:14:54.027963+0000","last_up_change":"2024-08-19T13:14:53.010492+0000","last_in_change":"2024-08-19T13:14:46.047417+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T13:14:54.027206+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01d9bd23-e3e8-4905-af40-142b43a27628","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6809","nonce":1798955708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6811","nonce":1798955708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6815","nonce":1798955708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6813","nonce":1798955708}]},"public_addr":"172.21.15.103:6809/1798955708","cluster_addr":"172.21.15.103:6811/1798955708","heartbeat_back_addr":"172.21.15.103:6815/1798955708","heartbeat_front_addr":"172.21.15.103:6813/1798955708","state":["exists","up"]},{"osd":1,"uuid":"d562d5f9-b0ac-4f9c-856d-cdcf1be3da18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6825","nonce":855529591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6827","nonce":855529591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6831","nonce":855529591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6829","nonce":855529591}]},"public_addr":"172.21.15.103:6825/855529591","cluster_addr":"172.21.15.103:6827/855529591","heartbeat_back_addr":"172.21.15.103:6831/855529591","heartbeat_front_addr":"172.21.15.103:6829/855529591","state":["exists","up"]},{"osd":2,"uuid":"adc47b95-8fbe-48c0-bb44-f78ba8ed6d29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6801","nonce":4004268210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6803","nonce":4004268210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6807","nonce":4004268210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6805","nonce":4004268210}]},"public_addr":"172.21.15.103:6801/4004268210","cluster_addr":"172.21.15.103:6803/4004268210","heartbeat_back_addr":"172.21.15.103:6807/4004268210","heartbeat_front_addr":"172.21.15.103:6805/4004268210","state":["exists","up"]},{"osd":3,"uuid":"a2e730c7-32c8-4387-bdee-e00a70e7613a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6817","nonce":264232662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6819","nonce":264232662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6823","nonce":264232662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6821","nonce":264232662}]},"public_addr":"172.21.15.103:6817/264232662","cluster_addr":"172.21.15.103:6819/264232662","heartbeat_back_addr":"172.21.15.103:6823/264232662","heartbeat_front_addr":"172.21.15.103:6821/264232662","state":["exists","up"]},{"osd":4,"uuid":"45b5502e-ce4c-411c-b7a2-1742483a432f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6801","nonce":634629443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6803","nonce":634629443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6807","nonce":634629443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6805","nonce":634629443}]},"public_addr":"172.21.15.122:6801/634629443","cluster_addr":"172.21.15.122:6803/634629443","heartbeat_back_addr":"172.21.15.122:6807/634629443","heartbeat_front_addr":"172.21.15.122:6805/634629443","state":["exists","up"]},{"osd":5,"uuid":"056397bb-673f-461f-87c8-e44390d7ba37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6817","nonce":2112974081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6819","nonce":2112974081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6823","nonce":2112974081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6821","nonce":2112974081}]},"public_addr":"172.21.15.122:6817/2112974081","cluster_addr":"172.21.15.122:6819/2112974081","heartbeat_back_addr":"172.21.15.122:6823/2112974081","heartbeat_front_addr":"172.21.15.122:6821/2112974081","state":["exists","up"]},{"osd":6,"uuid":"fab2d2d2-c670-4f56-b099-ca4686395179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6809","nonce":2033312647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6811","nonce":2033312647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6815","nonce":2033312647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6813","nonce":2033312647}]},"public_addr":"172.21.15.122:6809/2033312647","cluster_addr":"172.21.15.122:6811/2033312647","heartbeat_back_addr":"172.21.15.122:6815/2033312647","heartbeat_front_addr":"172.21.15.122:6813/2033312647","state":["exists","up"]},{"osd":7,"uuid":"ee58594c-242f-436e-9c9e-977e57b6785f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6825","nonce":3566117434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6827","nonce":3566117434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6831","nonce":3566117434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6829","nonce":3566117434}]},"public_addr":"172.21.15.122:6825/3566117434","cluster_addr":"172.21.15.122:6827/3566117434","heartbeat_back_addr":"172.21.15.122:6831/3566117434","heartbeat_front_addr":"172.21.15.122:6829/3566117434","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.192109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.184240+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.258392+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.144042+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.137671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.217965+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.194891+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.241799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T13:14:54.388 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.387+0000 7f0434813640 1 -- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f041006bda0 msgr2=0x7f041006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:54.388 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.387+0000 7f0434813640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f041006bda0 0x7f041006e260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0420009b70 tx=0x7f0420008040 comp rx=0 tx=0).stop 2024-08-19T13:14:54.388 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.387+0000 7f0434813640 1 -- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0430072610 msgr2=0x7f0430146360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:54.388 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.387+0000 7f0434813640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0430072610 0x7f0430146360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f04180098c0 tx=0x7f04180046c0 comp rx=0 tx=0).stop 2024-08-19T13:14:54.389 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.387+0000 7f0434813640 1 -- 172.21.15.103:0/147735325 shutdown_connections 2024-08-19T13:14:54.389 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.387+0000 7f0434813640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f041006bda0 0x7f041006e260 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:54.389 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.387+0000 7f0434813640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0430072a30 0x7f0430143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:54.389 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.387+0000 7f0434813640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0430072610 0x7f0430146360 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:54.389 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.387+0000 7f0434813640 1 --2- 172.21.15.103:0/147735325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f043006cbc0 0x7f0430147450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:54.389 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.387+0000 7f0434813640 1 -- 172.21.15.103:0/147735325 >> 172.21.15.103:0/147735325 conn(0x7f04300b1340 msgr2=0x7f04300b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:54.389 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.387+0000 7f0434813640 1 -- 172.21.15.103:0/147735325 shutdown_connections 2024-08-19T13:14:54.389 INFO:teuthology.misc.health.smithi103.stderr:2024-08-19T13:14:54.387+0000 7f0434813640 1 -- 172.21.15.103:0/147735325 wait complete. 2024-08-19T13:14:54.399 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-08-19T13:14:54.399 INFO:tasks.ceph:Creating RBD pool 2024-08-19T13:14:54.399 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd pool create rbd 8 2024-08-19T13:14:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.496+0000 7fb7db798640 1 Processor -- start 2024-08-19T13:14:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.497+0000 7fb7db798640 1 -- start start 2024-08-19T13:14:54.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.497+0000 7fb7db798640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7d406bae0 0x7fb7d4143b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:54.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.497+0000 7fb7db798640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7d4071870 con 0x7fb7d41440c0 2024-08-19T13:14:54.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.497+0000 7fb7db798640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7d4144550 con 0x7fb7d4070810 2024-08-19T13:14:54.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.497+0000 7fb7db798640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb7d406bec0 con 0x7fb7d406bae0 2024-08-19T13:14:54.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.497+0000 7fb7d9534640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7d406bae0 0x7fb7d4143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:54.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.497+0000 7fb7d9d35640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fb7d41440c0 0x7fb7d41511d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:41898/0 (socket says 172.21.15.103:41898) 2024-08-19T13:14:54.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.497+0000 7fb7d9d35640 1 -- 172.21.15.103:0/1657075890 learned_addr learned my addr 172.21.15.103:0/1657075890 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:14:54.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.498+0000 7fb7d9534640 1 -- 172.21.15.103:0/1657075890 >> v1:172.21.15.122:6789/0 conn(0x7fb7d4070810 legacy=0x7fb7d414a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:54.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.498+0000 7fb7d9534640 1 -- 172.21.15.103:0/1657075890 >> v1:172.21.15.103:6789/0 conn(0x7fb7d41440c0 legacy=0x7fb7d41511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:54.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.498+0000 7fb7d9534640 1 -- 172.21.15.103:0/1657075890 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d4071ae0 con 0x7fb7d406bae0 2024-08-19T13:14:54.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.498+0000 7fb7cbfff640 1 -- 172.21.15.103:0/1657075890 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 500284818 0 0) 0x7fb7d4071870 con 0x7fb7d41440c0 2024-08-19T13:14:54.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.498+0000 7fb7d9534640 1 --2- 172.21.15.103:0/1657075890 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7d406bae0 0x7fb7d4143b80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00097d0 tx=0x7fb7d002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea2c95b201811dcb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:54.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.498+0000 7fb7cbfff640 1 -- 172.21.15.103:0/1657075890 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0037070 con 0x7fb7d406bae0 2024-08-19T13:14:54.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.498+0000 7fb7cbfff640 1 -- 172.21.15.103:0/1657075890 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d002ae40 con 0x7fb7d406bae0 2024-08-19T13:14:54.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.498+0000 7fb7cbfff640 1 -- 172.21.15.103:0/1657075890 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0032660 con 0x7fb7d406bae0 2024-08-19T13:14:54.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.499+0000 7fb7db798640 1 -- 172.21.15.103:0/1657075890 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7d406bae0 msgr2=0x7fb7d4143b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:54.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.499+0000 7fb7db798640 1 --2- 172.21.15.103:0/1657075890 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7d406bae0 0x7fb7d4143b80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00097d0 tx=0x7fb7d002a130 comp rx=0 tx=0).stop 2024-08-19T13:14:54.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.499+0000 7fb7db798640 1 -- 172.21.15.103:0/1657075890 shutdown_connections 2024-08-19T13:14:54.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.499+0000 7fb7db798640 1 --2- 172.21.15.103:0/1657075890 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7d406bae0 0x7fb7d4143b80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:54.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.499+0000 7fb7db798640 1 -- 172.21.15.103:0/1657075890 >> 172.21.15.103:0/1657075890 conn(0x7fb7d40b5030 msgr2=0x7fb7d40b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.499+0000 7fb7db798640 1 -- 172.21.15.103:0/1657075890 shutdown_connections 2024-08-19T13:14:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.499+0000 7fb7db798640 1 -- 172.21.15.103:0/1657075890 wait complete. 2024-08-19T13:14:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.500+0000 7fb7db798640 1 Processor -- start 2024-08-19T13:14:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.500+0000 7fb7db798640 1 -- start start 2024-08-19T13:14:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7db798640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7d406bae0 0x7fb7d40b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7db798640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7d41440c0 0x7fb7d4144720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7db798640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7d40a94c0 0x7fb7d41dc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7db798640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d414a830 con 0x7fb7d41440c0 2024-08-19T13:14:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7db798640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d40700e0 con 0x7fb7d40a94c0 2024-08-19T13:14:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7db798640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb7d414a9b0 con 0x7fb7d406bae0 2024-08-19T13:14:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7d9534640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7d406bae0 0x7fb7d40b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7d8d33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7d41440c0 0x7fb7d4144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7d9534640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7d406bae0 0x7fb7d40b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:46256/0 (socket says 172.21.15.103:46256) 2024-08-19T13:14:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7d9534640 1 -- 172.21.15.103:0/3217479127 learned_addr learned my addr 172.21.15.103:0/3217479127 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7d9d35640 1 --2- 172.21.15.103:0/3217479127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7d40a94c0 0x7fb7d41dc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7d8d33640 1 -- 172.21.15.103:0/3217479127 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7d406bae0 msgr2=0x7fb7d40b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7d8d33640 1 --2- 172.21.15.103:0/3217479127 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7d406bae0 0x7fb7d40b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7d8d33640 1 -- 172.21.15.103:0/3217479127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7d40a94c0 msgr2=0x7fb7d41dc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7d8d33640 1 --2- 172.21.15.103:0/3217479127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7d40a94c0 0x7fb7d41dc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7d8d33640 1 -- 172.21.15.103:0/3217479127 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d40aa700 con 0x7fb7d41440c0 2024-08-19T13:14:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7d9d35640 1 --2- 172.21.15.103:0/3217479127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7d40a94c0 0x7fb7d41dc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T13:14:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7d8d33640 1 --2- 172.21.15.103:0/3217479127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7d41440c0 0x7fb7d4144720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400c440 tx=0x7fb7c400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:54.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7c9ffb640 1 -- 172.21.15.103:0/3217479127 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4011790 con 0x7fb7d41440c0 2024-08-19T13:14:54.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7c9ffb640 1 -- 172.21.15.103:0/3217479127 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7c4017070 con 0x7fb7d41440c0 2024-08-19T13:14:54.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7db798640 1 -- 172.21.15.103:0/3217479127 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d40aa420 con 0x7fb7d41440c0 2024-08-19T13:14:54.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.501+0000 7fb7c9ffb640 1 -- 172.21.15.103:0/3217479127 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4011b70 con 0x7fb7d41440c0 2024-08-19T13:14:54.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.502+0000 7fb7db798640 1 -- 172.21.15.103:0/3217479127 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d41474d0 con 0x7fb7d41440c0 2024-08-19T13:14:54.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.502+0000 7fb7db798640 1 -- 172.21.15.103:0/3217479127 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d4143b80 con 0x7fb7d41440c0 2024-08-19T13:14:54.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.503+0000 7fb7c9ffb640 1 -- 172.21.15.103:0/3217479127 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fb7c4011d10 con 0x7fb7d41440c0 2024-08-19T13:14:54.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.503+0000 7fb7c9ffb640 1 --2- 172.21.15.103:0/3217479127 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fb7b806bdd0 0x7fb7b806e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:54.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.503+0000 7fb7c9ffb640 1 -- 172.21.15.103:0/3217479127 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 4846+0+0 (secure 0 0 0) 0x7fb7c408a6f0 con 0x7fb7d41440c0 2024-08-19T13:14:54.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.503+0000 7fb7d9534640 1 --2- 172.21.15.103:0/3217479127 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fb7b806bdd0 0x7fb7b806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:54.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.504+0000 7fb7d9534640 1 --2- 172.21.15.103:0/3217479127 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fb7b806bdd0 0x7fb7b806e290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0000c00 tx=0x7fb7d002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:54.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.508+0000 7fb7c9ffb640 1 -- 172.21.15.103:0/3217479127 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fb7c40594e0 con 0x7fb7d41440c0 2024-08-19T13:14:54.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:54.671+0000 7fb7db798640 1 -- 172.21.15.103:0/3217479127 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7fb7d41471a0 con 0x7fb7d41440c0 2024-08-19T13:14:55.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.034+0000 7fb7c9ffb640 1 -- 172.21.15.103:0/3217479127 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) ==== 109+0+0 (secure 0 0 0) 0x7fb7c4058de0 con 0x7fb7d41440c0 2024-08-19T13:14:55.035 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'rbd' created 2024-08-19T13:14:55.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.037+0000 7fb7db798640 1 -- 172.21.15.103:0/3217479127 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fb7b806bdd0 msgr2=0x7fb7b806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:55.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.037+0000 7fb7db798640 1 --2- 172.21.15.103:0/3217479127 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fb7b806bdd0 0x7fb7b806e290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0000c00 tx=0x7fb7d002e000 comp rx=0 tx=0).stop 2024-08-19T13:14:55.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.037+0000 7fb7db798640 1 -- 172.21.15.103:0/3217479127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7d41440c0 msgr2=0x7fb7d4144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:55.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.037+0000 7fb7db798640 1 --2- 172.21.15.103:0/3217479127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7d41440c0 0x7fb7d4144720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400c440 tx=0x7fb7c400c900 comp rx=0 tx=0).stop 2024-08-19T13:14:55.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.037+0000 7fb7db798640 1 -- 172.21.15.103:0/3217479127 shutdown_connections 2024-08-19T13:14:55.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.037+0000 7fb7db798640 1 --2- 172.21.15.103:0/3217479127 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fb7b806bdd0 0x7fb7b806e290 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:55.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.037+0000 7fb7db798640 1 --2- 172.21.15.103:0/3217479127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7d40a94c0 0x7fb7d41dc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:55.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.037+0000 7fb7db798640 1 --2- 172.21.15.103:0/3217479127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7d41440c0 0x7fb7d4144720 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:55.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.037+0000 7fb7db798640 1 --2- 172.21.15.103:0/3217479127 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7d406bae0 0x7fb7d40b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:55.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.037+0000 7fb7db798640 1 -- 172.21.15.103:0/3217479127 >> 172.21.15.103:0/3217479127 conn(0x7fb7d40b5030 msgr2=0x7fb7d40b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:55.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.037+0000 7fb7db798640 1 -- 172.21.15.103:0/3217479127 shutdown_connections 2024-08-19T13:14:55.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.038+0000 7fb7db798640 1 -- 172.21.15.103:0/3217479127 wait complete. 2024-08-19T13:14:55.052 DEBUG:teuthology.orchestra.run.smithi103:> rbd --cluster ceph pool init rbd 2024-08-19T13:14:55.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.089+0000 7fa73b62dd80 1 Processor -- start 2024-08-19T13:14:55.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.089+0000 7fa73b62dd80 1 -- start start 2024-08-19T13:14:55.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.090+0000 7fa73b62dd80 1 --2- >> v2:172.21.15.103:3300/0 conn(0x55e3b8f83040 0x55e3b8f804c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:55.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.090+0000 7fa73b62dd80 1 --2- >> v2:172.21.15.122:3300/0 conn(0x55e3b90132b0 0x55e3b8f84c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:55.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.090+0000 7fa73b62dd80 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55e3b8dd6ca0 con 0x55e3b8f851d0 2024-08-19T13:14:55.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.090+0000 7fa73b62dd80 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x55e3b8de04d0 con 0x55e3b8f83040 2024-08-19T13:14:55.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.090+0000 7fa73b62dd80 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x55e3b8e44ef0 con 0x55e3b90132b0 2024-08-19T13:14:55.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.090+0000 7fa738e26640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x55e3b8f83040 0x55e3b8f804c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:55.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.090+0000 7fa738e26640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x55e3b8f83040 0x55e3b8f804c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52694/0 (socket says 172.21.15.103:52694) 2024-08-19T13:14:55.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.090+0000 7fa738e26640 1 -- 172.21.15.103:0/2412118141 learned_addr learned my addr 172.21.15.103:0/2412118141 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:55.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.090+0000 7fa733fff640 1 --2- 172.21.15.103:0/2412118141 >> v2:172.21.15.122:3300/0 conn(0x55e3b90132b0 0x55e3b8f84c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:55.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.090+0000 7fa738e26640 1 -- 172.21.15.103:0/2412118141 >> v2:172.21.15.122:3300/0 conn(0x55e3b90132b0 msgr2=0x55e3b8f84c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:55.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.090+0000 7fa738e26640 1 --2- 172.21.15.103:0/2412118141 >> v2:172.21.15.122:3300/0 conn(0x55e3b90132b0 0x55e3b8f84c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:55.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.090+0000 7fa738e26640 1 -- 172.21.15.103:0/2412118141 >> v1:172.21.15.122:6789/0 conn(0x55e3b8f851d0 legacy=0x55e3b9016830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:55.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.090+0000 7fa738e26640 1 -- 172.21.15.103:0/2412118141 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b8dcf4f0 con 0x55e3b8f83040 2024-08-19T13:14:55.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.092+0000 7fa738e26640 1 --2- 172.21.15.103:0/2412118141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3b8f83040 0x55e3b8f804c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x55e3b8f7fb20 tx=0x7fa72c045ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41756a4901555dd8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:55.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.092+0000 7fa7337fe640 1 -- 172.21.15.103:0/2412118141 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c056020 con 0x55e3b8f83040 2024-08-19T13:14:55.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.092+0000 7fa7337fe640 1 -- 172.21.15.103:0/2412118141 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa72c052640 con 0x55e3b8f83040 2024-08-19T13:14:55.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.092+0000 7fa7337fe640 1 -- 172.21.15.103:0/2412118141 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c04e040 con 0x55e3b8f83040 2024-08-19T13:14:55.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.093+0000 7fa73b62dd80 1 -- 172.21.15.103:0/2412118141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3b8f83040 msgr2=0x55e3b8f804c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:55.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.093+0000 7fa73b62dd80 1 --2- 172.21.15.103:0/2412118141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3b8f83040 0x55e3b8f804c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x55e3b8f7fb20 tx=0x7fa72c045ac0 comp rx=0 tx=0).stop 2024-08-19T13:14:55.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.093+0000 7fa73b62dd80 1 -- 172.21.15.103:0/2412118141 shutdown_connections 2024-08-19T13:14:55.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.093+0000 7fa73b62dd80 1 --2- 172.21.15.103:0/2412118141 >> v2:172.21.15.122:3300/0 conn(0x55e3b90132b0 0x55e3b8f84c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:55.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.093+0000 7fa73b62dd80 1 --2- 172.21.15.103:0/2412118141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3b8f83040 0x55e3b8f804c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:55.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.093+0000 7fa73b62dd80 1 -- 172.21.15.103:0/2412118141 >> 172.21.15.103:0/2412118141 conn(0x55e3b8f85c60 msgr2=0x55e3b8f86070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:55.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.093+0000 7fa73b62dd80 1 -- 172.21.15.103:0/2412118141 shutdown_connections 2024-08-19T13:14:55.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.093+0000 7fa73b62dd80 1 -- 172.21.15.103:0/2412118141 wait complete. 2024-08-19T13:14:55.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.094+0000 7fa73b62dd80 1 Processor -- start 2024-08-19T13:14:55.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.094+0000 7fa73b62dd80 1 -- start start 2024-08-19T13:14:55.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.094+0000 7fa73b62dd80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3b8f83040 0x55e3b8f87780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:55.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.094+0000 7fa73b62dd80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3b8f851d0 0x55e3b9026890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:55.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.094+0000 7fa73b62dd80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e3b90132b0 0x55e3b9026dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:55.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.094+0000 7fa73b62dd80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e3b8debc40 con 0x55e3b8f851d0 2024-08-19T13:14:55.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.094+0000 7fa73b62dd80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e3b8e40eb0 con 0x55e3b8f83040 2024-08-19T13:14:55.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.094+0000 7fa73b62dd80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x55e3b8f80a00 con 0x55e3b90132b0 2024-08-19T13:14:55.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.094+0000 7fa733fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3b8f851d0 0x55e3b9026890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:55.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.094+0000 7fa733fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3b8f851d0 0x55e3b9026890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52702/0 (socket says 172.21.15.103:52702) 2024-08-19T13:14:55.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.094+0000 7fa73a088640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e3b90132b0 0x55e3b9026dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:55.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.094+0000 7fa733fff640 1 -- 172.21.15.103:0/1650791641 learned_addr learned my addr 172.21.15.103:0/1650791641 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:55.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.095+0000 7fa738e26640 1 --2- 172.21.15.103:0/1650791641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3b8f83040 0x55e3b8f87780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:55.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.095+0000 7fa733fff640 1 -- 172.21.15.103:0/1650791641 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e3b90132b0 msgr2=0x55e3b9026dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:55.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.095+0000 7fa733fff640 1 --2- 172.21.15.103:0/1650791641 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e3b90132b0 0x55e3b9026dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:55.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.095+0000 7fa733fff640 1 -- 172.21.15.103:0/1650791641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3b8f83040 msgr2=0x55e3b8f87780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:55.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.095+0000 7fa733fff640 1 --2- 172.21.15.103:0/1650791641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3b8f83040 0x55e3b8f87780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:55.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.095+0000 7fa733fff640 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b8dc5ce0 con 0x55e3b8f851d0 2024-08-19T13:14:55.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.095+0000 7fa733fff640 1 --2- 172.21.15.103:0/1650791641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3b8f851d0 0x55e3b9026890 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa724008700 tx=0x7fa7240087f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:55.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.095+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa724012790 con 0x55e3b8f851d0 2024-08-19T13:14:55.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.095+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa724018070 con 0x55e3b8f851d0 2024-08-19T13:14:55.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.095+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa724012b30 con 0x55e3b8f851d0 2024-08-19T13:14:55.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.095+0000 7fa73b62dd80 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e3b8f7c630 con 0x55e3b8f851d0 2024-08-19T13:14:55.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.095+0000 7fa73b62dd80 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b8f7b4f0 con 0x55e3b8f851d0 2024-08-19T13:14:55.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.097+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fa724012cd0 con 0x55e3b8f851d0 2024-08-19T13:14:55.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.097+0000 7fa730ff9640 1 --2- 172.21.15.103:0/1650791641 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa71c06a690 0x7fa71c06cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:55.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.097+0000 7fa738e26640 1 --2- 172.21.15.103:0/1650791641 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa71c06a690 0x7fa71c06cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:55.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.097+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 5199+0+0 (secure 0 0 0) 0x7fa724089720 con 0x55e3b8f851d0 2024-08-19T13:14:55.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.097+0000 7fa73b62dd80 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) -- 0x55e3b8f7a920 con 0x55e3b8f851d0 2024-08-19T13:14:55.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:55.098+0000 7fa738e26640 1 --2- 172.21.15.103:0/1650791641 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa71c06a690 0x7fa71c06cb50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa72c057000 tx=0x7fa72c052000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:56.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.035+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v16) ==== 141+0+0 (secure 0 0 0) 0x7fa724057df0 con 0x55e3b8f851d0 2024-08-19T13:14:56.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.035+0000 7fa73b62dd80 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa72c052640 con 0x55e3b8f851d0 2024-08-19T13:14:56.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.040+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fa724054090 con 0x55e3b8f851d0 2024-08-19T13:14:56.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.041+0000 7fa732ffd640 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=16}) -- 0x7fa714002ae0 con 0x55e3b8f851d0 2024-08-19T13:14:56.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.041+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 <== mon.0 v2:172.21.15.103:3300/0 8 ==== osd_map(16..16 src has 1..16) ==== 1036+0+0 (secure 0 0 0) 0x7fa72406d070 con 0x55e3b8f851d0 2024-08-19T13:14:56.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.041+0000 7fa73b62dd80 1 --2- 172.21.15.103:0/1650791641 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x55e3b900f9b0 0x55e3b8f568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:56.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.041+0000 7fa73b62dd80 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16) -- 0x55e3b8f56e10 con 0x55e3b900f9b0 2024-08-19T13:14:56.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.041+0000 7fa73a088640 1 --2- 172.21.15.103:0/1650791641 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x55e3b900f9b0 0x55e3b8f568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:56.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.041+0000 7fa73a088640 1 --2- 172.21.15.103:0/1650791641 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x55e3b900f9b0 0x55e3b8f568d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:56.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.050+0000 7fa73a088640 1 -- 172.21.15.103:0/1650791641 <== osd.5 v2:172.21.15.122:6816/2112974081 1 ==== osd_op_reply(1 rbd_trash [create] v16'1 uv1 ondisk = 0) ==== 153+0+0 (crc 0 0 0) 0x7fa734002630 con 0x55e3b900f9b0 2024-08-19T13:14:56.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.052+0000 7fa73b62dd80 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16) -- 0x55e3b9081640 con 0x55e3b900f9b0 2024-08-19T13:14:56.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.052+0000 7fa73a088640 1 -- 172.21.15.103:0/1650791641 <== osd.5 v2:172.21.15.122:6816/2112974081 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7fa734002630 con 0x55e3b900f9b0 2024-08-19T13:14:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.052+0000 7fa73b62dd80 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16) -- 0x55e3b9080750 con 0x55e3b900f9b0 2024-08-19T13:14:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.053+0000 7fa73a088640 1 -- 172.21.15.103:0/1650791641 <== osd.5 v2:172.21.15.122:6816/2112974081 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7fa734002630 con 0x55e3b900f9b0 2024-08-19T13:14:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.053+0000 7fa7327fc640 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) -- 0x7fa718001f00 con 0x55e3b8f851d0 2024-08-19T13:14:56.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:56.933+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 <== mon.0 v2:172.21.15.103:3300/0 9 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa72406f050 con 0x55e3b8f851d0 2024-08-19T13:14:57.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:57.051+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 <== mon.0 v2:172.21.15.103:3300/0 10 ==== pool_op_reply(tid 4 (0) Success v17) ==== 55+0+0 (secure 0 0 0) 0x7fa72403d1c0 con 0x55e3b8f851d0 2024-08-19T13:14:57.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:57.051+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=17}) -- 0x7fa71c034fc0 con 0x55e3b8f851d0 2024-08-19T13:14:57.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:57.051+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 <== mon.0 v2:172.21.15.103:3300/0 11 ==== osd_map(17..17 src has 1..17) ==== 626+0+0 (secure 0 0 0) 0x7fa724055e90 con 0x55e3b8f851d0 2024-08-19T13:14:57.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:57.051+0000 7fa732ffd640 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17) -- 0x7fa714005680 con 0x55e3b900f9b0 2024-08-19T13:14:57.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:57.054+0000 7fa73a088640 1 -- 172.21.15.103:0/1650791641 <== osd.5 v2:172.21.15.122:6816/2112974081 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v17'1 uv1 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fa734002630 con 0x55e3b900f9b0 2024-08-19T13:14:57.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:57.054+0000 7fa7327fc640 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v17) -- 0x7fa718003350 con 0x55e3b8f851d0 2024-08-19T13:14:58.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.056+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 <== mon.0 v2:172.21.15.103:3300/0 12 ==== pool_op_reply(tid 6 (0) Success v18) ==== 43+0+0 (secure 0 0 0) 0x7fa72405c0f0 con 0x55e3b8f851d0 2024-08-19T13:14:58.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.056+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=18}) -- 0x7fa71c035190 con 0x55e3b8f851d0 2024-08-19T13:14:58.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.057+0000 7fa730ff9640 1 -- 172.21.15.103:0/1650791641 <== mon.0 v2:172.21.15.103:3300/0 13 ==== osd_map(18..18 src has 1..18) ==== 654+0+0 (secure 0 0 0) 0x7fa7240560f0 con 0x55e3b8f851d0 2024-08-19T13:14:58.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.057+0000 7fa732ffd640 1 -- 172.21.15.103:0/1650791641 --> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) -- 0x7fa714005a20 con 0x55e3b900f9b0 2024-08-19T13:14:58.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.061+0000 7fa73a088640 1 -- 172.21.15.103:0/1650791641 <== osd.5 v2:172.21.15.122:6816/2112974081 5 ==== osd_op_reply(7 rbd_info [write 0~19] v18'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa734002630 con 0x55e3b900f9b0 2024-08-19T13:14:58.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.062+0000 7fa73b62dd80 1 -- 172.21.15.103:0/1650791641 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x55e3b900f9b0 msgr2=0x55e3b8f568d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.062+0000 7fa73b62dd80 1 --2- 172.21.15.103:0/1650791641 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x55e3b900f9b0 0x55e3b8f568d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.062+0000 7fa73b62dd80 1 -- 172.21.15.103:0/1650791641 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa71c06a690 msgr2=0x7fa71c06cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.062+0000 7fa73b62dd80 1 --2- 172.21.15.103:0/1650791641 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa71c06a690 0x7fa71c06cb50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa72c057000 tx=0x7fa72c052000 comp rx=0 tx=0).stop 2024-08-19T13:14:58.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.062+0000 7fa73b62dd80 1 -- 172.21.15.103:0/1650791641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3b8f851d0 msgr2=0x55e3b9026890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.062+0000 7fa73b62dd80 1 --2- 172.21.15.103:0/1650791641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3b8f851d0 0x55e3b9026890 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa724008700 tx=0x7fa7240087f0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.062+0000 7fa73a088640 1 -- 172.21.15.103:0/1650791641 reap_dead start 2024-08-19T13:14:58.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.062+0000 7fa73b62dd80 1 -- 172.21.15.103:0/1650791641 shutdown_connections 2024-08-19T13:14:58.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.062+0000 7fa73b62dd80 1 -- 172.21.15.103:0/1650791641 >> 172.21.15.103:0/1650791641 conn(0x55e3b8f85c60 msgr2=0x55e3b8f574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:58.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.062+0000 7fa73b62dd80 1 -- 172.21.15.103:0/1650791641 shutdown_connections 2024-08-19T13:14:58.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.062+0000 7fa73b62dd80 1 -- 172.21.15.103:0/1650791641 wait complete. 2024-08-19T13:14:58.075 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-08-19T13:14:58.076 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-19T13:14:58.076 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-08-19T13:14:58.076 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-08-19T13:14:58.076 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-08-19T13:14:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.226+0000 7fa2adfd6640 1 Processor -- start 2024-08-19T13:14:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.226+0000 7fa2adfd6640 1 -- start start 2024-08-19T13:14:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.226+0000 7fa2adfd6640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2a80a9d70 0x7fa2a80aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.226+0000 7fa2adfd6640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa2a80af150 0x7fa2a813fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.226+0000 7fa2adfd6640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a80abd50 con 0x7fa2a80aa150 2024-08-19T13:14:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.226+0000 7fa2adfd6640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a80aae50 con 0x7fa2a80af150 2024-08-19T13:14:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.226+0000 7fa2adfd6640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a80af5a0 con 0x7fa2a80a9d70 2024-08-19T13:14:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.227+0000 7fa2a77fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2a80a9d70 0x7fa2a80aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.227+0000 7fa2a7fff640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa2a80af150 0x7fa2a813fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.227+0000 7fa2a7fff640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa2a80af150 0x7fa2a813fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49706/0 (socket says 172.21.15.103:49706) 2024-08-19T13:14:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.227+0000 7fa2a77fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2a80a9d70 0x7fa2a80aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48478/0 (socket says 172.21.15.103:48478) 2024-08-19T13:14:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.227+0000 7fa2a7fff640 1 -- 172.21.15.103:0/3569573552 learned_addr learned my addr 172.21.15.103:0/3569573552 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.227+0000 7fa2a7fff640 1 -- 172.21.15.103:0/3569573552 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2a80a9d70 msgr2=0x7fa2a80aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.227+0000 7fa2a7fff640 1 --2- 172.21.15.103:0/3569573552 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2a80a9d70 0x7fa2a80aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.227+0000 7fa2a7fff640 1 -- 172.21.15.103:0/3569573552 >> v1:172.21.15.122:6789/0 conn(0x7fa2a80aa150 legacy=0x7fa2a813f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.227+0000 7fa2a7fff640 1 -- 172.21.15.103:0/3569573552 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a80abfc0 con 0x7fa2a80af150 2024-08-19T13:14:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.227+0000 7fa2a77fe640 1 --2- 172.21.15.103:0/3569573552 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2a80a9d70 0x7fa2a80aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:58.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.228+0000 7fa2a7fff640 1 --2- 172.21.15.103:0/3569573552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a80af150 0x7fa2a813fbc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009e50 tx=0x7fa2a002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cb7062fb517fad server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:58.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.228+0000 7fa2a6ffd640 1 -- 172.21.15.103:0/3569573552 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0036070 con 0x7fa2a80af150 2024-08-19T13:14:58.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.228+0000 7fa2a6ffd640 1 -- 172.21.15.103:0/3569573552 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a0031600 con 0x7fa2a80af150 2024-08-19T13:14:58.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.228+0000 7fa2a6ffd640 1 -- 172.21.15.103:0/3569573552 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00319d0 con 0x7fa2a80af150 2024-08-19T13:14:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.228+0000 7fa2adfd6640 1 -- 172.21.15.103:0/3569573552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a80af150 msgr2=0x7fa2a813fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.228+0000 7fa2adfd6640 1 --2- 172.21.15.103:0/3569573552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a80af150 0x7fa2a813fbc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009e50 tx=0x7fa2a002a6c0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.228+0000 7fa2adfd6640 1 -- 172.21.15.103:0/3569573552 shutdown_connections 2024-08-19T13:14:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.228+0000 7fa2adfd6640 1 --2- 172.21.15.103:0/3569573552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a80af150 0x7fa2a813fbc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.228+0000 7fa2adfd6640 1 --2- 172.21.15.103:0/3569573552 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2a80a9d70 0x7fa2a80aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.228+0000 7fa2adfd6640 1 -- 172.21.15.103:0/3569573552 >> 172.21.15.103:0/3569573552 conn(0x7fa2a80b4ec0 msgr2=0x7fa2a80b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.229+0000 7fa2adfd6640 1 -- 172.21.15.103:0/3569573552 shutdown_connections 2024-08-19T13:14:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.229+0000 7fa2adfd6640 1 -- 172.21.15.103:0/3569573552 wait complete. 2024-08-19T13:14:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.229+0000 7fa2adfd6640 1 Processor -- start 2024-08-19T13:14:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.230+0000 7fa2adfd6640 1 -- start start 2024-08-19T13:14:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.230+0000 7fa2adfd6640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2a80a9d70 0x7fa2a8148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.230+0000 7fa2adfd6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a80aa150 0x7fa2a8070820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.230+0000 7fa2adfd6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2a80af150 0x7fa2a8071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.230+0000 7fa2adfd6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8073960 con 0x7fa2a80aa150 2024-08-19T13:14:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.230+0000 7fa2adfd6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8149c10 con 0x7fa2a80af150 2024-08-19T13:14:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.230+0000 7fa2adfd6640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a80728c0 con 0x7fa2a80a9d70 2024-08-19T13:14:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.230+0000 7fa2a77fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2a80a9d70 0x7fa2a8148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.230+0000 7fa29ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a80aa150 0x7fa2a8070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.230+0000 7fa2a77fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2a80a9d70 0x7fa2a8148260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48482/0 (socket says 172.21.15.103:48482) 2024-08-19T13:14:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.230+0000 7fa2a77fe640 1 -- 172.21.15.103:0/648058979 learned_addr learned my addr 172.21.15.103:0/648058979 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.230+0000 7fa2a77fe640 1 -- 172.21.15.103:0/648058979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2a80af150 msgr2=0x7fa2a8071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.230+0000 7fa2a77fe640 1 --2- 172.21.15.103:0/648058979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2a80af150 0x7fa2a8071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.231+0000 7fa2a77fe640 1 -- 172.21.15.103:0/648058979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a80aa150 msgr2=0x7fa2a8070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.231+0000 7fa2a77fe640 1 --2- 172.21.15.103:0/648058979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a80aa150 0x7fa2a8070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.231+0000 7fa2a77fe640 1 -- 172.21.15.103:0/648058979 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a8072ba0 con 0x7fa2a80a9d70 2024-08-19T13:14:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.231+0000 7fa29ffff640 1 --2- 172.21.15.103:0/648058979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a80aa150 0x7fa2a8070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.231+0000 7fa2a77fe640 1 --2- 172.21.15.103:0/648058979 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2a80a9d70 0x7fa2a8148260 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa29800ec40 tx=0x7fa29800c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.231+0000 7fa2a4ff9640 1 -- 172.21.15.103:0/648058979 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298017070 con 0x7fa2a80a9d70 2024-08-19T13:14:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.231+0000 7fa2a4ff9640 1 -- 172.21.15.103:0/648058979 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa29800cc30 con 0x7fa2a80a9d70 2024-08-19T13:14:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.231+0000 7fa2adfd6640 1 -- 172.21.15.103:0/648058979 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a81db1f0 con 0x7fa2a80a9d70 2024-08-19T13:14:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.231+0000 7fa2a4ff9640 1 -- 172.21.15.103:0/648058979 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298013470 con 0x7fa2a80a9d70 2024-08-19T13:14:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.231+0000 7fa2adfd6640 1 -- 172.21.15.103:0/648058979 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a8071b50 con 0x7fa2a80a9d70 2024-08-19T13:14:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.232+0000 7fa2adfd6640 1 -- 172.21.15.103:0/648058979 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a80aec10 con 0x7fa2a80a9d70 2024-08-19T13:14:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.232+0000 7fa2a4ff9640 1 -- 172.21.15.103:0/648058979 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa29801a070 con 0x7fa2a80a9d70 2024-08-19T13:14:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.233+0000 7fa2a4ff9640 1 --2- 172.21.15.103:0/648058979 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa28806be30 0x7fa28806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.233+0000 7fa29ffff640 1 --2- 172.21.15.103:0/648058979 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa28806be30 0x7fa28806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.233+0000 7fa2a4ff9640 1 -- 172.21.15.103:0/648058979 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7fa29808ab40 con 0x7fa2a80a9d70 2024-08-19T13:14:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.233+0000 7fa29ffff640 1 --2- 172.21.15.103:0/648058979 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa28806be30 0x7fa28806e2f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa2a81d8db0 tx=0x7fa290007e30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.237+0000 7fa2a4ff9640 1 -- 172.21.15.103:0/648058979 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fa2980590e0 con 0x7fa2a80a9d70 2024-08-19T13:14:58.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.450+0000 7fa2adfd6640 1 -- 172.21.15.103:0/648058979 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa2a80ac3c0 con 0x7fa2a80a9d70 2024-08-19T13:14:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.453+0000 7fa2a4ff9640 1 -- 172.21.15.103:0/648058979 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172945 (secure 0 0 0) 0x7fa29805d3e0 con 0x7fa2a80a9d70 2024-08-19T13:14:58.456 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.455+0000 7fa2adfd6640 1 -- 172.21.15.103:0/648058979 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa28806be30 msgr2=0x7fa28806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.455+0000 7fa2adfd6640 1 --2- 172.21.15.103:0/648058979 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa28806be30 0x7fa28806e2f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa2a81d8db0 tx=0x7fa290007e30 comp rx=0 tx=0).stop 2024-08-19T13:14:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.455+0000 7fa2adfd6640 1 -- 172.21.15.103:0/648058979 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2a80a9d70 msgr2=0x7fa2a8148260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.455+0000 7fa2adfd6640 1 --2- 172.21.15.103:0/648058979 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2a80a9d70 0x7fa2a8148260 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa29800ec40 tx=0x7fa29800c520 comp rx=0 tx=0).stop 2024-08-19T13:14:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.455+0000 7fa2adfd6640 1 -- 172.21.15.103:0/648058979 shutdown_connections 2024-08-19T13:14:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.455+0000 7fa2adfd6640 1 --2- 172.21.15.103:0/648058979 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa28806be30 0x7fa28806e2f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.455+0000 7fa2adfd6640 1 --2- 172.21.15.103:0/648058979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2a80af150 0x7fa2a8071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.455+0000 7fa2adfd6640 1 --2- 172.21.15.103:0/648058979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a80aa150 0x7fa2a8070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.455+0000 7fa2adfd6640 1 --2- 172.21.15.103:0/648058979 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2a80a9d70 0x7fa2a8148260 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.455+0000 7fa2adfd6640 1 -- 172.21.15.103:0/648058979 >> 172.21.15.103:0/648058979 conn(0x7fa2a80b4ec0 msgr2=0x7fa2a80b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.455+0000 7fa2adfd6640 1 -- 172.21.15.103:0/648058979 shutdown_connections 2024-08-19T13:14:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.455+0000 7fa2adfd6640 1 -- 172.21.15.103:0/648058979 wait complete. 2024-08-19T13:14:58.468 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":5,"flags":0,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3235664023},{"type":"v1","addr":"172.21.15.103:6833","nonce":3235664023}]},"active_addr":"172.21.15.103:6833/3235664023","active_change":"2024-08-19T13:14:47.988663+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":2935071724}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":107970887}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":4212566882}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":1902099607}]}]} 2024-08-19T13:14:58.473 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-08-19T13:14:58.473 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-08-19T13:14:58.473 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-19T13:14:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.571+0000 7f244dffa640 1 Processor -- start 2024-08-19T13:14:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.571+0000 7f244dffa640 1 -- start start 2024-08-19T13:14:58.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.572+0000 7f244dffa640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f24480aec80 0x7f24480731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.572+0000 7f244dffa640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2448070360 0x7f244813c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.572+0000 7f244dffa640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24480ae490 con 0x7f2448073710 2024-08-19T13:14:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.572+0000 7f244dffa640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f24480af060 con 0x7f24480aec80 2024-08-19T13:14:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.572+0000 7f244dffa640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f244806da70 con 0x7f2448070360 2024-08-19T13:14:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.572+0000 7f243ffff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2448070360 0x7f244813c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.572+0000 7f243ffff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2448070360 0x7f244813c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48486/0 (socket says 172.21.15.103:48486) 2024-08-19T13:14:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.572+0000 7f243ffff640 1 -- 172.21.15.103:0/2204434873 learned_addr learned my addr 172.21.15.103:0/2204434873 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.572+0000 7f24477fe640 1 --2- 172.21.15.103:0/2204434873 >> v2:172.21.15.122:3300/0 conn(0x7f24480aec80 0x7f24480731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.572+0000 7f243ffff640 1 -- 172.21.15.103:0/2204434873 >> v2:172.21.15.122:3300/0 conn(0x7f24480aec80 msgr2=0x7f24480731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.572+0000 7f243ffff640 1 --2- 172.21.15.103:0/2204434873 >> v2:172.21.15.122:3300/0 conn(0x7f24480aec80 0x7f24480731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.572+0000 7f243ffff640 1 -- 172.21.15.103:0/2204434873 >> v1:172.21.15.122:6789/0 conn(0x7f2448073710 legacy=0x7f244813c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.572+0000 7f243ffff640 1 -- 172.21.15.103:0/2204434873 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24480ad910 con 0x7f2448070360 2024-08-19T13:14:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.573+0000 7f243ffff640 1 --2- 172.21.15.103:0/2204434873 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2448070360 0x7f244813c3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2430009870 tx=0x7f243002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa69b6e75b2a1169 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.573+0000 7f2446ffd640 1 -- 172.21.15.103:0/2204434873 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430037070 con 0x7f2448070360 2024-08-19T13:14:58.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.573+0000 7f2446ffd640 1 -- 172.21.15.103:0/2204434873 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2430032440 con 0x7f2448070360 2024-08-19T13:14:58.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.573+0000 7f2446ffd640 1 -- 172.21.15.103:0/2204434873 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430032810 con 0x7f2448070360 2024-08-19T13:14:58.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.574+0000 7f244dffa640 1 -- 172.21.15.103:0/2204434873 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2448070360 msgr2=0x7f244813c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.574+0000 7f244dffa640 1 --2- 172.21.15.103:0/2204434873 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2448070360 0x7f244813c3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2430009870 tx=0x7f243002a180 comp rx=0 tx=0).stop 2024-08-19T13:14:58.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.574+0000 7f244dffa640 1 -- 172.21.15.103:0/2204434873 shutdown_connections 2024-08-19T13:14:58.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.574+0000 7f244dffa640 1 --2- 172.21.15.103:0/2204434873 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2448070360 0x7f244813c3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.574+0000 7f244dffa640 1 --2- 172.21.15.103:0/2204434873 >> v2:172.21.15.122:3300/0 conn(0x7f24480aec80 0x7f24480731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.574+0000 7f244dffa640 1 -- 172.21.15.103:0/2204434873 >> 172.21.15.103:0/2204434873 conn(0x7f24480b41c0 msgr2=0x7f24480b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:58.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.574+0000 7f244dffa640 1 -- 172.21.15.103:0/2204434873 shutdown_connections 2024-08-19T13:14:58.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.574+0000 7f244dffa640 1 -- 172.21.15.103:0/2204434873 wait complete. 2024-08-19T13:14:58.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.575+0000 7f244dffa640 1 Processor -- start 2024-08-19T13:14:58.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.575+0000 7f244dffa640 1 -- start start 2024-08-19T13:14:58.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.575+0000 7f244dffa640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2448070360 0x7f2448143050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.575+0000 7f244dffa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2448073710 0x7f2448137f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.575+0000 7f244dffa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24480aec80 0x7f24481384c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.575+0000 7f244dffa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2448149bd0 con 0x7f24480aec80 2024-08-19T13:14:58.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.575+0000 7f244dffa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2448149a50 con 0x7f2448073710 2024-08-19T13:14:58.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.575+0000 7f244dffa640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f2448149d50 con 0x7f2448070360 2024-08-19T13:14:58.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.575+0000 7f24477fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2448070360 0x7f2448143050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.575+0000 7f24477fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2448070360 0x7f2448143050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48490/0 (socket says 172.21.15.103:48490) 2024-08-19T13:14:58.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.575+0000 7f24477fe640 1 -- 172.21.15.103:0/1318308964 learned_addr learned my addr 172.21.15.103:0/1318308964 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:58.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.575+0000 7f2447fff640 1 --2- 172.21.15.103:0/1318308964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24480aec80 0x7f24481384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.575+0000 7f243ffff640 1 --2- 172.21.15.103:0/1318308964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2448073710 0x7f2448137f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.576+0000 7f24477fe640 1 -- 172.21.15.103:0/1318308964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2448073710 msgr2=0x7f2448137f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.576+0000 7f24477fe640 1 --2- 172.21.15.103:0/1318308964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2448073710 0x7f2448137f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.576+0000 7f24477fe640 1 -- 172.21.15.103:0/1318308964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24480aec80 msgr2=0x7f24481384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.576+0000 7f24477fe640 1 --2- 172.21.15.103:0/1318308964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24480aec80 0x7f24481384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.576+0000 7f24477fe640 1 -- 172.21.15.103:0/1318308964 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24481372c0 con 0x7f2448070360 2024-08-19T13:14:58.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.576+0000 7f24477fe640 1 --2- 172.21.15.103:0/1318308964 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2448070360 0x7f2448143050 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f243800d5f0 tx=0x7f243800dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:58.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.576+0000 7f2444ff9640 1 -- 172.21.15.103:0/1318308964 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380125f0 con 0x7f2448070360 2024-08-19T13:14:58.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.576+0000 7f2444ff9640 1 -- 172.21.15.103:0/1318308964 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2438019070 con 0x7f2448070360 2024-08-19T13:14:58.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.576+0000 7f244dffa640 1 -- 172.21.15.103:0/1318308964 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24481375a0 con 0x7f2448070360 2024-08-19T13:14:58.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.576+0000 7f2444ff9640 1 -- 172.21.15.103:0/1318308964 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380129d0 con 0x7f2448070360 2024-08-19T13:14:58.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.576+0000 7f244dffa640 1 -- 172.21.15.103:0/1318308964 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2448137b60 con 0x7f2448070360 2024-08-19T13:14:58.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.577+0000 7f244dffa640 1 -- 172.21.15.103:0/1318308964 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24480731d0 con 0x7f2448070360 2024-08-19T13:14:58.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.577+0000 7f2444ff9640 1 -- 172.21.15.103:0/1318308964 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2438012b70 con 0x7f2448070360 2024-08-19T13:14:58.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.578+0000 7f2444ff9640 1 --2- 172.21.15.103:0/1318308964 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f242806be60 0x7f242806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.578+0000 7f243ffff640 1 --2- 172.21.15.103:0/1318308964 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f242806be60 0x7f242806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.578+0000 7f2444ff9640 1 -- 172.21.15.103:0/1318308964 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f243808af70 con 0x7f2448070360 2024-08-19T13:14:58.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.578+0000 7f243ffff640 1 --2- 172.21.15.103:0/1318308964 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f242806be60 0x7f242806e320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2430034000 tx=0x7f243002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:58.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.582+0000 7f2444ff9640 1 -- 172.21.15.103:0/1318308964 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f2438059480 con 0x7f2448070360 2024-08-19T13:14:58.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.749+0000 7f244dffa640 1 -- 172.21.15.103:0/1318308964 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2448075c30 con 0x7f2448070360 2024-08-19T13:14:58.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.750+0000 7f2444ff9640 1 -- 172.21.15.103:0/1318308964 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+14835 (secure 0 0 0) 0x7f243805d780 con 0x7f2448070360 2024-08-19T13:14:58.750 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:58.750 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":18,"fsid":"9a6e456b-d04b-48cb-846f-0fbd1ee79f13","created":"2024-08-19T13:14:43.897750+0000","modified":"2024-08-19T13:14:58.048300+0000","last_up_change":"2024-08-19T13:14:53.010492+0000","last_in_change":"2024-08-19T13:14:46.047417+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T13:14:54.027206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-08-19T13:14:54.689859+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01d9bd23-e3e8-4905-af40-142b43a27628","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6809","nonce":1798955708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6811","nonce":1798955708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6815","nonce":1798955708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6813","nonce":1798955708}]},"public_addr":"172.21.15.103:6809/1798955708","cluster_addr":"172.21.15.103:6811/1798955708","heartbeat_back_addr":"172.21.15.103:6815/1798955708","heartbeat_front_addr":"172.21.15.103:6813/1798955708","state":["exists","up"]},{"osd":1,"uuid":"d562d5f9-b0ac-4f9c-856d-cdcf1be3da18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6825","nonce":855529591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6827","nonce":855529591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6831","nonce":855529591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6829","nonce":855529591}]},"public_addr":"172.21.15.103:6825/855529591","cluster_addr":"172.21.15.103:6827/855529591","heartbeat_back_addr":"172.21.15.103:6831/855529591","heartbeat_front_addr":"172.21.15.103:6829/855529591","state":["exists","up"]},{"osd":2,"uuid":"adc47b95-8fbe-48c0-bb44-f78ba8ed6d29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6801","nonce":4004268210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6803","nonce":4004268210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6807","nonce":4004268210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6805","nonce":4004268210}]},"public_addr":"172.21.15.103:6801/4004268210","cluster_addr":"172.21.15.103:6803/4004268210","heartbeat_back_addr":"172.21.15.103:6807/4004268210","heartbeat_front_addr":"172.21.15.103:6805/4004268210","state":["exists","up"]},{"osd":3,"uuid":"a2e730c7-32c8-4387-bdee-e00a70e7613a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6817","nonce":264232662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6819","nonce":264232662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6823","nonce":264232662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6821","nonce":264232662}]},"public_addr":"172.21.15.103:6817/264232662","cluster_addr":"172.21.15.103:6819/264232662","heartbeat_back_addr":"172.21.15.103:6823/264232662","heartbeat_front_addr":"172.21.15.103:6821/264232662","state":["exists","up"]},{"osd":4,"uuid":"45b5502e-ce4c-411c-b7a2-1742483a432f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6801","nonce":634629443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6803","nonce":634629443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6807","nonce":634629443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6805","nonce":634629443}]},"public_addr":"172.21.15.122:6801/634629443","cluster_addr":"172.21.15.122:6803/634629443","heartbeat_back_addr":"172.21.15.122:6807/634629443","heartbeat_front_addr":"172.21.15.122:6805/634629443","state":["exists","up"]},{"osd":5,"uuid":"056397bb-673f-461f-87c8-e44390d7ba37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6817","nonce":2112974081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6819","nonce":2112974081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6823","nonce":2112974081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6821","nonce":2112974081}]},"public_addr":"172.21.15.122:6817/2112974081","cluster_addr":"172.21.15.122:6819/2112974081","heartbeat_back_addr":"172.21.15.122:6823/2112974081","heartbeat_front_addr":"172.21.15.122:6821/2112974081","state":["exists","up"]},{"osd":6,"uuid":"fab2d2d2-c670-4f56-b099-ca4686395179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6809","nonce":2033312647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6811","nonce":2033312647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6815","nonce":2033312647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6813","nonce":2033312647}]},"public_addr":"172.21.15.122:6809/2033312647","cluster_addr":"172.21.15.122:6811/2033312647","heartbeat_back_addr":"172.21.15.122:6815/2033312647","heartbeat_front_addr":"172.21.15.122:6813/2033312647","state":["exists","up"]},{"osd":7,"uuid":"ee58594c-242f-436e-9c9e-977e57b6785f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6825","nonce":3566117434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6827","nonce":3566117434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6831","nonce":3566117434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6829","nonce":3566117434}]},"public_addr":"172.21.15.122:6825/3566117434","cluster_addr":"172.21.15.122:6827/3566117434","heartbeat_back_addr":"172.21.15.122:6831/3566117434","heartbeat_front_addr":"172.21.15.122:6829/3566117434","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.192109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.184240+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.258392+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.144042+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.137671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.217965+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.194891+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.241799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T13:14:58.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.752+0000 7f244dffa640 1 -- 172.21.15.103:0/1318308964 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f242806be60 msgr2=0x7f242806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.752+0000 7f244dffa640 1 --2- 172.21.15.103:0/1318308964 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f242806be60 0x7f242806e320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2430034000 tx=0x7f243002e000 comp rx=0 tx=0).stop 2024-08-19T13:14:58.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.752+0000 7f244dffa640 1 -- 172.21.15.103:0/1318308964 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2448070360 msgr2=0x7f2448143050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.752+0000 7f244dffa640 1 --2- 172.21.15.103:0/1318308964 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2448070360 0x7f2448143050 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f243800d5f0 tx=0x7f243800dab0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.752+0000 7f244dffa640 1 -- 172.21.15.103:0/1318308964 shutdown_connections 2024-08-19T13:14:58.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.752+0000 7f244dffa640 1 --2- 172.21.15.103:0/1318308964 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f242806be60 0x7f242806e320 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.752+0000 7f244dffa640 1 --2- 172.21.15.103:0/1318308964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24480aec80 0x7f24481384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.752+0000 7f244dffa640 1 --2- 172.21.15.103:0/1318308964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2448073710 0x7f2448137f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.752+0000 7f244dffa640 1 --2- 172.21.15.103:0/1318308964 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2448070360 0x7f2448143050 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.752+0000 7f244dffa640 1 -- 172.21.15.103:0/1318308964 >> 172.21.15.103:0/1318308964 conn(0x7f24480b41c0 msgr2=0x7f2448149340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:58.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.752+0000 7f244dffa640 1 -- 172.21.15.103:0/1318308964 shutdown_connections 2024-08-19T13:14:58.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.752+0000 7f244dffa640 1 -- 172.21.15.103:0/1318308964 wait complete. 2024-08-19T13:14:58.763 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-08-19T13:14:58.763 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-19T13:14:58.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.860+0000 7fe7691d6640 1 Processor -- start 2024-08-19T13:14:58.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.860+0000 7fe7691d6640 1 -- start start 2024-08-19T13:14:58.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.861+0000 7fe7691d6640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe7640b04d0 0x7fe7641428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.861+0000 7fe7691d6640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe7640af8c0 0x7fe764142df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.861+0000 7fe7691d6640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe764143330 con 0x7fe76406ba70 2024-08-19T13:14:58.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.861+0000 7fe7691d6640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fe76406be50 con 0x7fe7640af8c0 2024-08-19T13:14:58.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.861+0000 7fe7691d6640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fe7640b1830 con 0x7fe7640b04d0 2024-08-19T13:14:58.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.861+0000 7fe76359e640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe7640af8c0 0x7fe764142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.861+0000 7fe76359e640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe7640af8c0 0x7fe764142df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49734/0 (socket says 172.21.15.103:49734) 2024-08-19T13:14:58.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.861+0000 7fe76359e640 1 -- 172.21.15.103:0/1056683757 learned_addr learned my addr 172.21.15.103:0/1056683757 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:58.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.861+0000 7fe75a59c640 1 --2- 172.21.15.103:0/1056683757 >> v2:172.21.15.122:3300/0 conn(0x7fe7640b04d0 0x7fe7641428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.861+0000 7fe76359e640 1 -- 172.21.15.103:0/1056683757 >> v2:172.21.15.122:3300/0 conn(0x7fe7640b04d0 msgr2=0x7fe7641428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:58.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.861+0000 7fe76359e640 1 --2- 172.21.15.103:0/1056683757 >> v2:172.21.15.122:3300/0 conn(0x7fe7640b04d0 0x7fe7641428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.861+0000 7fe76359e640 1 -- 172.21.15.103:0/1056683757 >> v1:172.21.15.122:6789/0 conn(0x7fe76406ba70 legacy=0x7fe76413f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.861+0000 7fe76359e640 1 -- 172.21.15.103:0/1056683757 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7640ad8d0 con 0x7fe7640af8c0 2024-08-19T13:14:58.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.861+0000 7fe75a59c640 1 --2- 172.21.15.103:0/1056683757 >> v2:172.21.15.122:3300/0 conn(0x7fe7640b04d0 0x7fe7641428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T13:14:58.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.862+0000 7fe76359e640 1 --2- 172.21.15.103:0/1056683757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7640af8c0 0x7fe764142df0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe7640740f0 tx=0x7fe75c02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2e122c44a18710 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:58.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.862+0000 7fe76259c640 1 -- 172.21.15.103:0/1056683757 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c036070 con 0x7fe7640af8c0 2024-08-19T13:14:58.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.862+0000 7fe76259c640 1 -- 172.21.15.103:0/1056683757 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75c031600 con 0x7fe7640af8c0 2024-08-19T13:14:58.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.862+0000 7fe76259c640 1 -- 172.21.15.103:0/1056683757 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0319d0 con 0x7fe7640af8c0 2024-08-19T13:14:58.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.863+0000 7fe7691d6640 1 -- 172.21.15.103:0/1056683757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7640af8c0 msgr2=0x7fe764142df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.863+0000 7fe7691d6640 1 --2- 172.21.15.103:0/1056683757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7640af8c0 0x7fe764142df0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe7640740f0 tx=0x7fe75c02a640 comp rx=0 tx=0).stop 2024-08-19T13:14:58.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.863+0000 7fe7691d6640 1 -- 172.21.15.103:0/1056683757 shutdown_connections 2024-08-19T13:14:58.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.863+0000 7fe7691d6640 1 --2- 172.21.15.103:0/1056683757 >> v2:172.21.15.122:3300/0 conn(0x7fe7640b04d0 0x7fe7641428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.863+0000 7fe7691d6640 1 --2- 172.21.15.103:0/1056683757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7640af8c0 0x7fe764142df0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.863+0000 7fe7691d6640 1 -- 172.21.15.103:0/1056683757 >> 172.21.15.103:0/1056683757 conn(0x7fe7640b50a0 msgr2=0x7fe7640b54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:58.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.863+0000 7fe7691d6640 1 -- 172.21.15.103:0/1056683757 shutdown_connections 2024-08-19T13:14:58.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.863+0000 7fe7691d6640 1 -- 172.21.15.103:0/1056683757 wait complete. 2024-08-19T13:14:58.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.864+0000 7fe7691d6640 1 Processor -- start 2024-08-19T13:14:58.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.864+0000 7fe7691d6640 1 -- start start 2024-08-19T13:14:58.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe7691d6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe76406ba70 0x7fe764072da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe7691d6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7640af8c0 0x7fe764147060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe7691d6640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7640b04d0 0x7fe76414b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe7691d6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe764140090 con 0x7fe76406ba70 2024-08-19T13:14:58.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe7691d6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe76413ff10 con 0x7fe7640af8c0 2024-08-19T13:14:58.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe7691d6640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe76414cde0 con 0x7fe7640b04d0 2024-08-19T13:14:58.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe762d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe76406ba70 0x7fe764072da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe76359e640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7640b04d0 0x7fe76414b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe76359e640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7640b04d0 0x7fe76414b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48502/0 (socket says 172.21.15.103:48502) 2024-08-19T13:14:58.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe76359e640 1 -- 172.21.15.103:0/2957097752 learned_addr learned my addr 172.21.15.103:0/2957097752 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:58.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe75a59c640 1 --2- 172.21.15.103:0/2957097752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7640af8c0 0x7fe764147060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe76359e640 1 -- 172.21.15.103:0/2957097752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7640af8c0 msgr2=0x7fe764147060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe76359e640 1 --2- 172.21.15.103:0/2957097752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7640af8c0 0x7fe764147060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe76359e640 1 -- 172.21.15.103:0/2957097752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe76406ba70 msgr2=0x7fe764072da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:58.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe76359e640 1 --2- 172.21.15.103:0/2957097752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe76406ba70 0x7fe764072da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:58.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe76359e640 1 -- 172.21.15.103:0/2957097752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7641476d0 con 0x7fe7640b04d0 2024-08-19T13:14:58.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe762d9d640 1 --2- 172.21.15.103:0/2957097752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe76406ba70 0x7fe764072da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:58.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.865+0000 7fe76359e640 1 --2- 172.21.15.103:0/2957097752 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7640b04d0 0x7fe76414b390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009f70 tx=0x7fe75c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:58.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.866+0000 7fe75bfff640 1 -- 172.21.15.103:0/2957097752 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c036050 con 0x7fe7640b04d0 2024-08-19T13:14:58.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.866+0000 7fe75bfff640 1 -- 172.21.15.103:0/2957097752 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75c042070 con 0x7fe7640b04d0 2024-08-19T13:14:58.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.866+0000 7fe7691d6640 1 -- 172.21.15.103:0/2957097752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe764148b20 con 0x7fe7640b04d0 2024-08-19T13:14:58.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.866+0000 7fe75bfff640 1 -- 172.21.15.103:0/2957097752 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c007bd0 con 0x7fe7640b04d0 2024-08-19T13:14:58.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.866+0000 7fe7691d6640 1 -- 172.21.15.103:0/2957097752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe76414b8d0 con 0x7fe7640b04d0 2024-08-19T13:14:58.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.867+0000 7fe7691d6640 1 -- 172.21.15.103:0/2957097752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe76413f800 con 0x7fe7640b04d0 2024-08-19T13:14:58.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.867+0000 7fe75bfff640 1 -- 172.21.15.103:0/2957097752 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe75c047460 con 0x7fe7640b04d0 2024-08-19T13:14:58.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.868+0000 7fe75bfff640 1 --2- 172.21.15.103:0/2957097752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe74406be30 0x7fe74406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:58.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.868+0000 7fe762d9d640 1 --2- 172.21.15.103:0/2957097752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe74406be30 0x7fe74406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:58.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.868+0000 7fe75bfff640 1 -- 172.21.15.103:0/2957097752 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7fe75c0aba30 con 0x7fe7640b04d0 2024-08-19T13:14:58.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.868+0000 7fe762d9d640 1 --2- 172.21.15.103:0/2957097752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe74406be30 0x7fe74406e2f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe754001b70 tx=0x7fe754009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:58.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:58.872+0000 7fe75bfff640 1 -- 172.21.15.103:0/2957097752 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fe75c02ae20 con 0x7fe7640b04d0 2024-08-19T13:14:59.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.040+0000 7fe7691d6640 1 -- 172.21.15.103:0/2957097752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe76414cbc0 con 0x7fe7640b04d0 2024-08-19T13:14:59.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.041+0000 7fe75bfff640 1 -- 172.21.15.103:0/2957097752 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+14835 (secure 0 0 0) 0x7fe75c079fd0 con 0x7fe7640b04d0 2024-08-19T13:14:59.041 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:14:59.041 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":18,"fsid":"9a6e456b-d04b-48cb-846f-0fbd1ee79f13","created":"2024-08-19T13:14:43.897750+0000","modified":"2024-08-19T13:14:58.048300+0000","last_up_change":"2024-08-19T13:14:53.010492+0000","last_in_change":"2024-08-19T13:14:46.047417+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T13:14:54.027206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-08-19T13:14:54.689859+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01d9bd23-e3e8-4905-af40-142b43a27628","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6809","nonce":1798955708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6811","nonce":1798955708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6815","nonce":1798955708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6813","nonce":1798955708}]},"public_addr":"172.21.15.103:6809/1798955708","cluster_addr":"172.21.15.103:6811/1798955708","heartbeat_back_addr":"172.21.15.103:6815/1798955708","heartbeat_front_addr":"172.21.15.103:6813/1798955708","state":["exists","up"]},{"osd":1,"uuid":"d562d5f9-b0ac-4f9c-856d-cdcf1be3da18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6825","nonce":855529591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6827","nonce":855529591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6831","nonce":855529591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6829","nonce":855529591}]},"public_addr":"172.21.15.103:6825/855529591","cluster_addr":"172.21.15.103:6827/855529591","heartbeat_back_addr":"172.21.15.103:6831/855529591","heartbeat_front_addr":"172.21.15.103:6829/855529591","state":["exists","up"]},{"osd":2,"uuid":"adc47b95-8fbe-48c0-bb44-f78ba8ed6d29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6801","nonce":4004268210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6803","nonce":4004268210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6807","nonce":4004268210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6805","nonce":4004268210}]},"public_addr":"172.21.15.103:6801/4004268210","cluster_addr":"172.21.15.103:6803/4004268210","heartbeat_back_addr":"172.21.15.103:6807/4004268210","heartbeat_front_addr":"172.21.15.103:6805/4004268210","state":["exists","up"]},{"osd":3,"uuid":"a2e730c7-32c8-4387-bdee-e00a70e7613a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6817","nonce":264232662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6819","nonce":264232662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6823","nonce":264232662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6821","nonce":264232662}]},"public_addr":"172.21.15.103:6817/264232662","cluster_addr":"172.21.15.103:6819/264232662","heartbeat_back_addr":"172.21.15.103:6823/264232662","heartbeat_front_addr":"172.21.15.103:6821/264232662","state":["exists","up"]},{"osd":4,"uuid":"45b5502e-ce4c-411c-b7a2-1742483a432f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6801","nonce":634629443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6803","nonce":634629443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6807","nonce":634629443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6805","nonce":634629443}]},"public_addr":"172.21.15.122:6801/634629443","cluster_addr":"172.21.15.122:6803/634629443","heartbeat_back_addr":"172.21.15.122:6807/634629443","heartbeat_front_addr":"172.21.15.122:6805/634629443","state":["exists","up"]},{"osd":5,"uuid":"056397bb-673f-461f-87c8-e44390d7ba37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6817","nonce":2112974081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6819","nonce":2112974081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6823","nonce":2112974081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6821","nonce":2112974081}]},"public_addr":"172.21.15.122:6817/2112974081","cluster_addr":"172.21.15.122:6819/2112974081","heartbeat_back_addr":"172.21.15.122:6823/2112974081","heartbeat_front_addr":"172.21.15.122:6821/2112974081","state":["exists","up"]},{"osd":6,"uuid":"fab2d2d2-c670-4f56-b099-ca4686395179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6809","nonce":2033312647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6811","nonce":2033312647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6815","nonce":2033312647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6813","nonce":2033312647}]},"public_addr":"172.21.15.122:6809/2033312647","cluster_addr":"172.21.15.122:6811/2033312647","heartbeat_back_addr":"172.21.15.122:6815/2033312647","heartbeat_front_addr":"172.21.15.122:6813/2033312647","state":["exists","up"]},{"osd":7,"uuid":"ee58594c-242f-436e-9c9e-977e57b6785f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6825","nonce":3566117434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6827","nonce":3566117434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6831","nonce":3566117434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6829","nonce":3566117434}]},"public_addr":"172.21.15.122:6825/3566117434","cluster_addr":"172.21.15.122:6827/3566117434","heartbeat_back_addr":"172.21.15.122:6831/3566117434","heartbeat_front_addr":"172.21.15.122:6829/3566117434","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.192109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.184240+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.258392+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.144042+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.137671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.217965+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.194891+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.241799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T13:14:59.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.043+0000 7fe7691d6640 1 -- 172.21.15.103:0/2957097752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe74406be30 msgr2=0x7fe74406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.043+0000 7fe7691d6640 1 --2- 172.21.15.103:0/2957097752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe74406be30 0x7fe74406e2f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe754001b70 tx=0x7fe754009040 comp rx=0 tx=0).stop 2024-08-19T13:14:59.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.043+0000 7fe7691d6640 1 -- 172.21.15.103:0/2957097752 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7640b04d0 msgr2=0x7fe76414b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.043+0000 7fe7691d6640 1 --2- 172.21.15.103:0/2957097752 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7640b04d0 0x7fe76414b390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009f70 tx=0x7fe75c004a40 comp rx=0 tx=0).stop 2024-08-19T13:14:59.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.043+0000 7fe7691d6640 1 -- 172.21.15.103:0/2957097752 shutdown_connections 2024-08-19T13:14:59.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.043+0000 7fe7691d6640 1 --2- 172.21.15.103:0/2957097752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe74406be30 0x7fe74406e2f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.043+0000 7fe7691d6640 1 --2- 172.21.15.103:0/2957097752 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7640b04d0 0x7fe76414b390 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.043+0000 7fe7691d6640 1 --2- 172.21.15.103:0/2957097752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7640af8c0 0x7fe764147060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.043+0000 7fe7691d6640 1 --2- 172.21.15.103:0/2957097752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe76406ba70 0x7fe764072da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.043+0000 7fe7691d6640 1 -- 172.21.15.103:0/2957097752 >> 172.21.15.103:0/2957097752 conn(0x7fe7640b50a0 msgr2=0x7fe7640b5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.043+0000 7fe7691d6640 1 -- 172.21.15.103:0/2957097752 shutdown_connections 2024-08-19T13:14:59.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.043+0000 7fe7691d6640 1 -- 172.21.15.103:0/2957097752 wait complete. 2024-08-19T13:14:59.055 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-08-19T13:14:59.055 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-08-19T13:14:59.055 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-08-19T13:14:59.056 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-08-19T13:14:59.056 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-08-19T13:14:59.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-08-19T13:14:59.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-08-19T13:14:59.058 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-08-19T13:14:59.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.221+0000 7f683002b640 1 Processor -- start 2024-08-19T13:14:59.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.222+0000 7f683002b640 1 -- start start 2024-08-19T13:14:59.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.222+0000 7ff2a1034640 1 Processor -- start 2024-08-19T13:14:59.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.222+0000 7f683002b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6828071740 0x7f68280aea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.223+0000 7f683002b640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6828073700 0x7f682813c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.223+0000 7f683002b640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f682813cb90 0x7f68280703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.223+0000 7f682d5c6640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6828073700 0x7f682813c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.223+0000 7f682d5c6640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6828073700 0x7f682813c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48516/0 (socket says 172.21.15.103:48516) 2024-08-19T13:14:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.223+0000 7f682d5c6640 1 -- 172.21.15.103:0/1309458242 learned_addr learned my addr 172.21.15.103:0/1309458242 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.223+0000 7f683002b640 1 -- 172.21.15.103:0/1309458242 --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f6828072f60 con 0x7f682813cb90 2024-08-19T13:14:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.223+0000 7f683002b640 1 -- 172.21.15.103:0/1309458242 --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f68280ae3d0 con 0x7f6828071740 2024-08-19T13:14:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.223+0000 7f683002b640 1 -- 172.21.15.103:0/1309458242 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f68280731e0 con 0x7f6828073700 2024-08-19T13:14:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.223+0000 7f682ddc7640 1 --2- 172.21.15.103:0/1309458242 >> v2:172.21.15.122:3300/0 conn(0x7f6828071740 0x7f68280aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.223+0000 7f682e5c8640 1 --2- 172.21.15.103:0/1309458242 >> v2:172.21.15.103:3300/0 conn(0x7f682813cb90 0x7f68280703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.223+0000 7ff2a1034640 1 -- start start 2024-08-19T13:14:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7ff2a1034640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff294034700 0x7ff2940cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7ff2a1034640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2940cbf00 con 0x7ff2940c8be0 2024-08-19T13:14:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7ff2a1034640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff294029f00 con 0x7ff294032870 2024-08-19T13:14:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7ff2a1034640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7ff294034ae0 con 0x7ff294034700 2024-08-19T13:14:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.223+0000 7f7e9ec15640 1 Processor -- start 2024-08-19T13:14:59.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.223+0000 7f7e9ec15640 1 -- start start 2024-08-19T13:14:59.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.223+0000 7f682e5c8640 1 -- 172.21.15.103:0/1309458242 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6828073700 msgr2=0x7f682813c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7f682e5c8640 1 --2- 172.21.15.103:0/1309458242 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6828073700 0x7f682813c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7f682e5c8640 1 -- 172.21.15.103:0/1309458242 >> v2:172.21.15.122:3300/0 conn(0x7f6828071740 msgr2=0x7f68280aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7f682e5c8640 1 --2- 172.21.15.103:0/1309458242 >> v2:172.21.15.122:3300/0 conn(0x7f6828071740 0x7f68280aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7f682e5c8640 1 -- 172.21.15.103:0/1309458242 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68280708e0 con 0x7f682813cb90 2024-08-19T13:14:59.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7f682d5c6640 1 --2- 172.21.15.103:0/1309458242 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6828073700 0x7f682813c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:59.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7f682ddc7640 1 --2- 172.21.15.103:0/1309458242 >> v2:172.21.15.122:3300/0 conn(0x7f6828071740 0x7f68280aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T13:14:59.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7ff29b7fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7ff294032870 0x7ff294029550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.103:59968/0 (socket says 172.21.15.103:59968) 2024-08-19T13:14:59.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7ff29b7fe640 1 -- 172.21.15.103:0/1086038415 learned_addr learned my addr 172.21.15.103:0/1086038415 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:14:59.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7ff29bfff640 1 --2- 172.21.15.103:0/1086038415 >> v2:172.21.15.103:3300/0 conn(0x7ff294034700 0x7ff2940cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7f7e9ec15640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7e9806c590 0x7f7e980a9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7f7e9ec15640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7e980afb70 0x7f7e98151360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7f7e9ec15640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e980abdb0 con 0x7f7e980a9e60 2024-08-19T13:14:59.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7f7e9ec15640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f7e9806c970 con 0x7f7e9806c590 2024-08-19T13:14:59.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.224+0000 7f7e9ec15640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7e980ac090 con 0x7f7e980afb70 2024-08-19T13:14:59.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7f7e9dc13640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7e9806c590 0x7f7e980a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7f7e9dc13640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7e9806c590 0x7f7e980a9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.103:48600/0 (socket says 172.21.15.103:48600) 2024-08-19T13:14:59.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7f7e9dc13640 1 -- 172.21.15.103:0/2215840646 learned_addr learned my addr 172.21.15.103:0/2215840646 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7f336c294640 1 Processor -- start 2024-08-19T13:14:59.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7ff29bfff640 1 -- 172.21.15.103:0/1086038415 >> v1:172.21.15.122:6789/0 conn(0x7ff294032870 legacy=0x7ff294029550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7ff29bfff640 1 -- 172.21.15.103:0/1086038415 >> v1:172.21.15.103:6789/0 conn(0x7ff2940c8be0 legacy=0x7ff2940c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7f7e9e414640 1 --2- 172.21.15.103:0/2215840646 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7e980afb70 0x7f7e98151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7ff29bfff640 1 -- 172.21.15.103:0/1086038415 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff294028c50 con 0x7ff294034700 2024-08-19T13:14:59.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7f682e5c8640 1 --2- 172.21.15.103:0/1309458242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f682813cb90 0x7f68280703a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f68280b1990 tx=0x7f682002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b141fd14019726cb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7f7e9dc13640 1 -- 172.21.15.103:0/2215840646 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7e980afb70 msgr2=0x7f7e98151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7f215fb13640 1 Processor -- start 2024-08-19T13:14:59.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7f7e9dc13640 1 --2- 172.21.15.103:0/2215840646 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7e980afb70 0x7f7e98151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7f7e9dc13640 1 -- 172.21.15.103:0/2215840646 >> v1:172.21.15.122:6789/0 conn(0x7f7e980a9e60 legacy=0x7f7e980af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7f7e9dc13640 1 -- 172.21.15.103:0/2215840646 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e980aa770 con 0x7f7e9806c590 2024-08-19T13:14:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7f7e9e414640 1 --2- 172.21.15.103:0/2215840646 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7e980afb70 0x7f7e98151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.226+0000 7fafb0325640 1 Processor -- start 2024-08-19T13:14:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.226+0000 7f682cdc5640 1 -- 172.21.15.103:0/1309458242 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820031600 con 0x7f682813cb90 2024-08-19T13:14:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.226+0000 7f682cdc5640 1 -- 172.21.15.103:0/1309458242 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68200317a0 con 0x7f682813cb90 2024-08-19T13:14:59.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.226+0000 7f682cdc5640 1 -- 172.21.15.103:0/1309458242 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820031b70 con 0x7f682813cb90 2024-08-19T13:14:59.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.225+0000 7f336c294640 1 -- start start 2024-08-19T13:14:59.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.226+0000 7fafb0325640 1 -- start start 2024-08-19T13:14:59.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.226+0000 7ff08df2c640 1 Processor -- start 2024-08-19T13:14:59.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.226+0000 7f215fb13640 1 -- start start 2024-08-19T13:14:59.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.226+0000 7ff29bfff640 1 --2- 172.21.15.103:0/1086038415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff294034700 0x7ff2940cb9c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff288009750 tx=0x7ff28802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2237c61a6f72d6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.226+0000 7f683002b640 1 -- 172.21.15.103:0/1309458242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f682813cb90 msgr2=0x7f68280703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f683002b640 1 --2- 172.21.15.103:0/1309458242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f682813cb90 0x7f68280703a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f68280b1990 tx=0x7f682002a5d0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f33c251d640 1 Processor -- start 2024-08-19T13:14:59.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f33c251d640 1 -- start start 2024-08-19T13:14:59.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f683002b640 1 -- 172.21.15.103:0/1309458242 shutdown_connections 2024-08-19T13:14:59.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f683002b640 1 --2- 172.21.15.103:0/1309458242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f682813cb90 0x7f68280703a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f683002b640 1 --2- 172.21.15.103:0/1309458242 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6828073700 0x7f682813c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f683002b640 1 --2- 172.21.15.103:0/1309458242 >> v2:172.21.15.122:3300/0 conn(0x7f6828071740 0x7f68280aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f683002b640 1 -- 172.21.15.103:0/1309458242 >> 172.21.15.103:0/1309458242 conn(0x7f68280b3fb0 msgr2=0x7f68280b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f336c294640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f335c02bfe0 0x7f335c0c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f336c294640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f335c036ea0 0x7f335c033160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f336c294640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f335c0c4860 con 0x7f335c033990 2024-08-19T13:14:59.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f336c294640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f335c02ad40 con 0x7f335c02bfe0 2024-08-19T13:14:59.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f336c294640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f335c02d200 con 0x7f335c036ea0 2024-08-19T13:14:59.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7fafb0325640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fafa806f4c0 0x7fafa8143bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7fafb0325640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa8071710 con 0x7fafa8144100 2024-08-19T13:14:59.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f336a030640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f335c02bfe0 0x7f335c0c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7fafb0325640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa80703f0 con 0x7fafa806f8a0 2024-08-19T13:14:59.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7ff29affd640 1 -- 172.21.15.103:0/1086038415 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288037070 con 0x7ff294034700 2024-08-19T13:14:59.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7ff29affd640 1 -- 172.21.15.103:0/1086038415 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff28802ad70 con 0x7ff294034700 2024-08-19T13:14:59.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f336a030640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f335c02bfe0 0x7f335c0c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.103:48602/0 (socket says 172.21.15.103:48602) 2024-08-19T13:14:59.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f336a030640 1 -- 172.21.15.103:0/3331200586 learned_addr learned my addr 172.21.15.103:0/3331200586 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7ff29affd640 1 -- 172.21.15.103:0/1086038415 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288032660 con 0x7ff294034700 2024-08-19T13:14:59.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.226+0000 7ff08df2c640 1 -- start start 2024-08-19T13:14:59.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7ff08df2c640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0880b4fa0 0x7ff0880b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7ff08df2c640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff088142330 0x7ff0880b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7ff08df2c640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0880a9880 con 0x7ff0880b14c0 2024-08-19T13:14:59.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7ff08df2c640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7ff0880b5380 con 0x7ff088142330 2024-08-19T13:14:59.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7ff08df2c640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff0880a94c0 con 0x7ff0880b4fa0 2024-08-19T13:14:59.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f215fb13640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2158072260 0x7f21580726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7fafb0325640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fafa8070760 con 0x7fafa806f4c0 2024-08-19T13:14:59.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f215fb13640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2158071bd0 con 0x7f21580ae7d0 2024-08-19T13:14:59.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f215fb13640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2158070890 con 0x7f215813b4d0 2024-08-19T13:14:59.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f215fb13640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f2158070630 con 0x7f2158072260 2024-08-19T13:14:59.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f215d8af640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f21580ae7d0 0x7f215806f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:51084/0 (socket says 172.21.15.103:51084) 2024-08-19T13:14:59.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f215d8af640 1 -- 172.21.15.103:0/3681668070 learned_addr learned my addr 172.21.15.103:0/3681668070 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:14:59.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7fafae0c1640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fafa806f4c0 0x7fafa8143bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7fafae0c1640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fafa806f4c0 0x7fafa8143bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49760/0 (socket says 172.21.15.103:49760) 2024-08-19T13:14:59.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7fafae0c1640 1 -- 172.21.15.103:0/1899288397 learned_addr learned my addr 172.21.15.103:0/1899288397 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f336a831640 1 --2- 172.21.15.103:0/3331200586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f335c036ea0 0x7f335c033160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f7e9dc13640 1 --2- 172.21.15.103:0/2215840646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9806c590 0x7f7e980a9920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009870 tx=0x7f7e9402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8758d5f79568d19 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f7e9cc11640 1 -- 172.21.15.103:0/2215840646 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94037070 con 0x7f7e9806c590 2024-08-19T13:14:59.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f7e9cc11640 1 -- 172.21.15.103:0/2215840646 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e94032440 con 0x7f7e9806c590 2024-08-19T13:14:59.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f7e9cc11640 1 -- 172.21.15.103:0/2215840646 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94032810 con 0x7f7e9806c590 2024-08-19T13:14:59.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7ff0877fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0880b4fa0 0x7ff0880b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7ff0877fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0880b4fa0 0x7ff0880b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48542/0 (socket says 172.21.15.103:48542) 2024-08-19T13:14:59.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7ff0877fe640 1 -- 172.21.15.103:0/234879209 learned_addr learned my addr 172.21.15.103:0/234879209 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7f336a030640 1 -- 172.21.15.103:0/3331200586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f335c036ea0 msgr2=0x7f335c033160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.227+0000 7f215e0b0640 1 --2- 172.21.15.103:0/3681668070 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2158072260 0x7f21580726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7f336902e640 1 -- 172.21.15.103:0/3331200586 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3411806672 0 0) 0x7f335c0c4860 con 0x7f335c033990 2024-08-19T13:14:59.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7f215e0b0640 1 -- 172.21.15.103:0/3681668070 >> v1:172.21.15.122:6789/0 conn(0x7f215813b4d0 legacy=0x7f215813b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7f215e0b0640 1 -- 172.21.15.103:0/3681668070 >> v1:172.21.15.103:6789/0 conn(0x7f21580ae7d0 legacy=0x7f215806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7ff2a1034640 1 -- 172.21.15.103:0/1086038415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff294034700 msgr2=0x7ff2940cb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7ff2a1034640 1 --2- 172.21.15.103:0/1086038415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff294034700 0x7ff2940cb9c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff288009750 tx=0x7ff28802a060 comp rx=0 tx=0).stop 2024-08-19T13:14:59.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7f7e9ec15640 1 -- 172.21.15.103:0/2215840646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9806c590 msgr2=0x7f7e980a9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7f7e9ec15640 1 --2- 172.21.15.103:0/2215840646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9806c590 0x7f7e980a9920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009870 tx=0x7f7e9402a180 comp rx=0 tx=0).stop 2024-08-19T13:14:59.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7f33c251d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f33bc13f490 0x7f33bc13f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7f33c251d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33bc14ae90 0x7f33bc14d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7fafae0c1640 1 -- 172.21.15.103:0/1899288397 >> v1:172.21.15.122:6789/0 conn(0x7fafa806f8a0 legacy=0x7fafa8146a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T13:14:59.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7fafae0c1640 1 -- 172.21.15.103:0/1899288397 >> v1:172.21.15.103:6789/0 conn(0x7fafa8144100 legacy=0x7fafa8147130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7fafae0c1640 1 -- 172.21.15.103:0/1899288397 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa81444e0 con 0x7fafa806f4c0 2024-08-19T13:14:59.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7f683002b640 1 -- 172.21.15.103:0/1309458242 shutdown_connections 2024-08-19T13:14:59.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7f33bbfff640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f33bc0aef70 0x7f33bc0af5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:51104/0 (socket says 172.21.15.103:51104) 2024-08-19T13:14:59.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7ff2a1034640 1 -- 172.21.15.103:0/1086038415 shutdown_connections 2024-08-19T13:14:59.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7ff2a1034640 1 --2- 172.21.15.103:0/1086038415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff294034700 0x7ff2940cb9c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7ff2a1034640 1 -- 172.21.15.103:0/1086038415 >> 172.21.15.103:0/1086038415 conn(0x7ff294005e20 msgr2=0x7ff294006230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f683002b640 1 -- 172.21.15.103:0/1309458242 wait complete. 2024-08-19T13:14:59.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7ff0877fe640 1 -- 172.21.15.103:0/234879209 >> v2:172.21.15.103:3300/0 conn(0x7ff088142330 msgr2=0x7ff0880b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7ff0877fe640 1 --2- 172.21.15.103:0/234879209 >> v2:172.21.15.103:3300/0 conn(0x7ff088142330 0x7ff0880b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7ff0867fc640 1 -- 172.21.15.103:0/234879209 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837327480 0 0) 0x7ff0880a9880 con 0x7ff0880b14c0 2024-08-19T13:14:59.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7ff0877fe640 1 -- 172.21.15.103:0/234879209 >> v1:172.21.15.103:6789/0 conn(0x7ff0880b14c0 legacy=0x7ff0880b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7ff0877fe640 1 -- 172.21.15.103:0/234879209 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0880b2390 con 0x7ff0880b4fa0 2024-08-19T13:14:59.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f33c251d640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33bc0b1790 con 0x7f33bc0aef70 2024-08-19T13:14:59.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f33c251d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f33bc0b4630 con 0x7f33bc13f490 2024-08-19T13:14:59.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7fafae0c1640 1 --2- 172.21.15.103:0/1899288397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafa806f4c0 0x7fafa8143bc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009870 tx=0x7fafa402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd199e7545ca16 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f33c251d640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f33bc1400d0 con 0x7f33bc14ae90 2024-08-19T13:14:59.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f33bbfff640 1 -- 172.21.15.103:0/2174694502 learned_addr learned my addr 172.21.15.103:0/2174694502 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:14:59.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7f336a030640 1 --2- 172.21.15.103:0/3331200586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f335c036ea0 0x7f335c033160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7f336a030640 1 -- 172.21.15.103:0/3331200586 >> v1:172.21.15.122:6789/0 conn(0x7f335c033990 legacy=0x7f335c02b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f33bb7fe640 1 --2- 172.21.15.103:0/2174694502 >> v2:172.21.15.122:3300/0 conn(0x7f33bc13f490 0x7f33bc13f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.228+0000 7f336a030640 1 -- 172.21.15.103:0/3331200586 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335c0336a0 con 0x7f335c02bfe0 2024-08-19T13:14:59.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f336a831640 1 --2- 172.21.15.103:0/3331200586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f335c036ea0 0x7f335c033160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T13:14:59.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f33c0aba640 1 --2- 172.21.15.103:0/2174694502 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33bc14ae90 0x7f33bc14d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f33baffd640 1 -- 172.21.15.103:0/2174694502 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 93212494 0 0) 0x7f33bc0b1790 con 0x7f33bc0aef70 2024-08-19T13:14:59.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f33baffd640 1 -- 172.21.15.103:0/2174694502 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33a4003520 con 0x7f33bc0aef70 2024-08-19T13:14:59.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7ff2a1034640 1 -- 172.21.15.103:0/1086038415 shutdown_connections 2024-08-19T13:14:59.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7ff2a1034640 1 -- 172.21.15.103:0/1086038415 wait complete. 2024-08-19T13:14:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f215e0b0640 1 -- 172.21.15.103:0/3681668070 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2158145e40 con 0x7f2158072260 2024-08-19T13:14:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f7e9ec15640 1 -- 172.21.15.103:0/2215840646 shutdown_connections 2024-08-19T13:14:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f7e9ec15640 1 --2- 172.21.15.103:0/2215840646 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7e980afb70 0x7f7e98151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f7e9ec15640 1 --2- 172.21.15.103:0/2215840646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9806c590 0x7f7e980a9920 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f7e9ec15640 1 -- 172.21.15.103:0/2215840646 >> 172.21.15.103:0/2215840646 conn(0x7f7e9806b800 msgr2=0x7f7e9806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f336a030640 1 --2- 172.21.15.103:0/3331200586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c02bfe0 0x7f335c0c4320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3360009870 tx=0x7f336002a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81035071fd232f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.230+0000 7fafad0bf640 1 -- 172.21.15.103:0/1899288397 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4037070 con 0x7fafa806f4c0 2024-08-19T13:14:59.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.230+0000 7fafad0bf640 1 -- 172.21.15.103:0/1899288397 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa4032440 con 0x7fafa806f4c0 2024-08-19T13:14:59.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.230+0000 7f336902e640 1 -- 172.21.15.103:0/3331200586 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360037070 con 0x7f335c02bfe0 2024-08-19T13:14:59.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.230+0000 7f336902e640 1 -- 172.21.15.103:0/3331200586 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3360032440 con 0x7f335c02bfe0 2024-08-19T13:14:59.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.230+0000 7f336902e640 1 -- 172.21.15.103:0/3331200586 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360032850 con 0x7f335c02bfe0 2024-08-19T13:14:59.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.230+0000 7ff0877fe640 1 --2- 172.21.15.103:0/234879209 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0880b4fa0 0x7ff0880b4430 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff078009870 tx=0x7ff07802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8746c5a651c6c0e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f33c0aba640 1 -- 172.21.15.103:0/2174694502 >> v2:172.21.15.122:3300/0 conn(0x7f33bc13f490 msgr2=0x7f33bc13f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f33c0aba640 1 --2- 172.21.15.103:0/2174694502 >> v2:172.21.15.122:3300/0 conn(0x7f33bc13f490 0x7f33bc13f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f33c0aba640 1 -- 172.21.15.103:0/2174694502 >> v1:172.21.15.103:6789/0 conn(0x7f33bc0aef70 legacy=0x7f33bc0af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f33c0aba640 1 -- 172.21.15.103:0/2174694502 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc0b1e70 con 0x7f33bc14ae90 2024-08-19T13:14:59.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.230+0000 7ff0867fc640 1 -- 172.21.15.103:0/234879209 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078037070 con 0x7ff0880b4fa0 2024-08-19T13:14:59.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.230+0000 7f215e0b0640 1 --2- 172.21.15.103:0/3681668070 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2158072260 0x7f21580726a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f214c0097d0 tx=0x7f214c02a120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be77ee366d938fd1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.230+0000 7fafad0bf640 1 -- 172.21.15.103:0/1899288397 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4037070 con 0x7fafa806f4c0 2024-08-19T13:14:59.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.229+0000 7f7e9ec15640 1 -- 172.21.15.103:0/2215840646 shutdown_connections 2024-08-19T13:14:59.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.230+0000 7f7e9ec15640 1 -- 172.21.15.103:0/2215840646 wait complete. 2024-08-19T13:14:59.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.230+0000 7f683002b640 1 Processor -- start 2024-08-19T13:14:59.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.230+0000 7ff2a1034640 1 Processor -- start 2024-08-19T13:14:59.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.230+0000 7f33c0aba640 1 --2- 172.21.15.103:0/2174694502 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33bc14ae90 0x7f33bc14d280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009d90 tx=0x7f33b402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce2c9d246d51b92 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7fafb0325640 1 -- 172.21.15.103:0/1899288397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafa806f4c0 msgr2=0x7fafa8143bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7fafb0325640 1 --2- 172.21.15.103:0/1899288397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafa806f4c0 0x7fafa8143bc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009870 tx=0x7fafa402a210 comp rx=0 tx=0).stop 2024-08-19T13:14:59.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.230+0000 7f683002b640 1 -- start start 2024-08-19T13:14:59.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7f33baffd640 1 -- 172.21.15.103:0/2174694502 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4007740 con 0x7f33bc14ae90 2024-08-19T13:14:59.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7ff2a1034640 1 -- start start 2024-08-19T13:14:59.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7f33baffd640 1 -- 172.21.15.103:0/2174694502 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b40078e0 con 0x7f33bc14ae90 2024-08-19T13:14:59.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7f215c8ad640 1 -- 172.21.15.103:0/3681668070 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c037070 con 0x7f2158072260 2024-08-19T13:14:59.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7f215c8ad640 1 -- 172.21.15.103:0/3681668070 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214c02ae30 con 0x7f2158072260 2024-08-19T13:14:59.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7f215c8ad640 1 -- 172.21.15.103:0/3681668070 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c032640 con 0x7f2158072260 2024-08-19T13:14:59.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7f33baffd640 1 -- 172.21.15.103:0/2174694502 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4007c90 con 0x7f33bc14ae90 2024-08-19T13:14:59.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7fafb0325640 1 -- 172.21.15.103:0/1899288397 shutdown_connections 2024-08-19T13:14:59.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7fafb0325640 1 --2- 172.21.15.103:0/1899288397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafa806f4c0 0x7fafa8143bc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7fafb0325640 1 -- 172.21.15.103:0/1899288397 >> 172.21.15.103:0/1899288397 conn(0x7fafa80b41c0 msgr2=0x7fafa80b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7fafb0325640 1 -- 172.21.15.103:0/1899288397 shutdown_connections 2024-08-19T13:14:59.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7ff2a1034640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff294032870 0x7ff2940c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7fafb0325640 1 -- 172.21.15.103:0/1899288397 wait complete. 2024-08-19T13:14:59.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7f683002b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6828071740 0x7f682813ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7f336c294640 1 -- 172.21.15.103:0/3331200586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c02bfe0 msgr2=0x7f335c0c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7f336c294640 1 --2- 172.21.15.103:0/3331200586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c02bfe0 0x7f335c0c4320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3360009870 tx=0x7f336002a1f0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f683002b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6828073700 0x7f682813b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7ff2a1034640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff294034700 0x7ff2940c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7ff2a1034640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2940c8be0 0x7ff2940c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7f7e9ec15640 1 Processor -- start 2024-08-19T13:14:59.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.231+0000 7f7e9ec15640 1 -- start start 2024-08-19T13:14:59.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7ff2a1034640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff2940d7470 con 0x7ff2940c8be0 2024-08-19T13:14:59.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f336c294640 1 -- 172.21.15.103:0/3331200586 shutdown_connections 2024-08-19T13:14:59.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f336c294640 1 --2- 172.21.15.103:0/3331200586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f335c036ea0 0x7f335c033160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f336c294640 1 --2- 172.21.15.103:0/3331200586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c02bfe0 0x7f335c0c4320 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f336c294640 1 -- 172.21.15.103:0/3331200586 >> 172.21.15.103:0/3331200586 conn(0x7f335c03a420 msgr2=0x7f335c036680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f33c251d640 1 -- 172.21.15.103:0/2174694502 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33bc14ae90 msgr2=0x7f33bc14d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f215fb13640 1 -- 172.21.15.103:0/3681668070 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2158072260 msgr2=0x7f21580726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f215fb13640 1 --2- 172.21.15.103:0/3681668070 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2158072260 0x7f21580726a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f214c0097d0 tx=0x7f214c02a120 comp rx=0 tx=0).stop 2024-08-19T13:14:59.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f33c251d640 1 --2- 172.21.15.103:0/2174694502 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33bc14ae90 0x7f33bc14d280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009d90 tx=0x7f33b402a5f0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7ff2a1034640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff2940290c0 con 0x7ff294032870 2024-08-19T13:14:59.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f7e9ec15640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7e9806c590 0x7f7e9811ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f7e9ec15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e980a9e60 0x7f7e98122d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f682d5c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6828073700 0x7f682813b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f682ddc7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6828071740 0x7f682813ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f682ddc7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6828071740 0x7f682813ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.103:48634/0 (socket says 172.21.15.103:48634) 2024-08-19T13:14:59.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f682d5c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6828073700 0x7f682813b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49778/0 (socket says 172.21.15.103:49778) 2024-08-19T13:14:59.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f682d5c6640 1 -- 172.21.15.103:0/2786204622 learned_addr learned my addr 172.21.15.103:0/2786204622 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7ff0867fc640 1 -- 172.21.15.103:0/234879209 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff078032440 con 0x7ff0880b4fa0 2024-08-19T13:14:59.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f683002b640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f682813cb90 0x7f68281d8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7ff2a1034640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff2940d75f0 con 0x7ff294034700 2024-08-19T13:14:59.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7ff29b7fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff294034700 0x7ff2940c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f683002b640 1 -- 172.21.15.103:0/2786204622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f682806f080 con 0x7f6828073700 2024-08-19T13:14:59.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f7e9d412640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e980a9e60 0x7f7e98122d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f7e9d412640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e980a9e60 0x7f7e98122d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49780/0 (socket says 172.21.15.103:49780) 2024-08-19T13:14:59.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f683002b640 1 -- 172.21.15.103:0/2786204622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f682806ef00 con 0x7f6828071740 2024-08-19T13:14:59.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f215fb13640 1 -- 172.21.15.103:0/3681668070 shutdown_connections 2024-08-19T13:14:59.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f215fb13640 1 --2- 172.21.15.103:0/3681668070 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2158072260 0x7f21580726a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f215fb13640 1 -- 172.21.15.103:0/3681668070 >> 172.21.15.103:0/3681668070 conn(0x7f21580b3fb0 msgr2=0x7f21580b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f7e9ec15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e980afb70 0x7f7e981e4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f7e9d412640 1 -- 172.21.15.103:0/4266759240 learned_addr learned my addr 172.21.15.103:0/4266759240 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f7e9ec15640 1 -- 172.21.15.103:0/4266759240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7e980aa2a0 con 0x7f7e980a9e60 2024-08-19T13:14:59.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f7e9ec15640 1 -- 172.21.15.103:0/4266759240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e980aafa0 con 0x7f7e980afb70 2024-08-19T13:14:59.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f7e9ec15640 1 -- 172.21.15.103:0/4266759240 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7e98068360 con 0x7f7e9806c590 2024-08-19T13:14:59.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f682e5c8640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f682813cb90 0x7f68281d8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f683002b640 1 -- 172.21.15.103:0/2786204622 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f682806f200 con 0x7f682813cb90 2024-08-19T13:14:59.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7ff29b7fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff294034700 0x7ff2940c4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48562/0 (socket says 172.21.15.103:48562) 2024-08-19T13:14:59.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7ff0867fc640 1 -- 172.21.15.103:0/234879209 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078032810 con 0x7ff0880b4fa0 2024-08-19T13:14:59.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7ff29b7fe640 1 -- 172.21.15.103:0/366360000 learned_addr learned my addr 172.21.15.103:0/366360000 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7ff2a0833640 1 --2- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2940c8be0 0x7ff2940c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7f215fb13640 1 -- 172.21.15.103:0/3681668070 shutdown_connections 2024-08-19T13:14:59.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7f336c294640 1 -- 172.21.15.103:0/3331200586 shutdown_connections 2024-08-19T13:14:59.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7fafb0325640 1 Processor -- start 2024-08-19T13:14:59.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7f336c294640 1 -- 172.21.15.103:0/3331200586 wait complete. 2024-08-19T13:14:59.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f7e9dc13640 1 --2- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7e9806c590 0x7f7e9811ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7f7e9d412640 1 -- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7e9806c590 msgr2=0x7f7e9811ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7f7e9d412640 1 --2- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7e9806c590 0x7f7e9811ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff29b7fe640 1 -- 172.21.15.103:0/366360000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff294032870 msgr2=0x7ff2940c82e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T13:14:59.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff29b7fe640 1 --2- 172.21.15.103:0/366360000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff294032870 0x7ff2940c82e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff29b7fe640 1 -- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2940c8be0 msgr2=0x7ff2940c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff29b7fe640 1 --2- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2940c8be0 0x7ff2940c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff29b7fe640 1 -- 172.21.15.103:0/366360000 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2940c4450 con 0x7ff294034700 2024-08-19T13:14:59.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7f7e9d412640 1 -- 172.21.15.103:0/4266759240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e980afb70 msgr2=0x7f7e981e4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff08df2c640 1 -- 172.21.15.103:0/234879209 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0880b4fa0 msgr2=0x7ff0880b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff08df2c640 1 --2- 172.21.15.103:0/234879209 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0880b4fa0 0x7ff0880b4430 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff078009870 tx=0x7ff07802a250 comp rx=0 tx=0).stop 2024-08-19T13:14:59.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7f7e9d412640 1 --2- 172.21.15.103:0/4266759240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e980afb70 0x7f7e981e4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7f7e9d412640 1 -- 172.21.15.103:0/4266759240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e98121d40 con 0x7f7e980a9e60 2024-08-19T13:14:59.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff08df2c640 1 -- 172.21.15.103:0/234879209 shutdown_connections 2024-08-19T13:14:59.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff08df2c640 1 --2- 172.21.15.103:0/234879209 >> v2:172.21.15.103:3300/0 conn(0x7ff088142330 0x7ff0880b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff08df2c640 1 --2- 172.21.15.103:0/234879209 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0880b4fa0 0x7ff0880b4430 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff08df2c640 1 -- 172.21.15.103:0/234879209 >> 172.21.15.103:0/234879209 conn(0x7ff088073110 msgr2=0x7ff088073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff29b7fe640 1 --2- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff294034700 0x7ff2940c4ec0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff29000c750 tx=0x7ff29000cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7f215fb13640 1 -- 172.21.15.103:0/3681668070 wait complete. 2024-08-19T13:14:59.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f33c251d640 1 -- 172.21.15.103:0/2174694502 shutdown_connections 2024-08-19T13:14:59.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f33c251d640 1 --2- 172.21.15.103:0/2174694502 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33bc14ae90 0x7f33bc14d280 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f33c251d640 1 --2- 172.21.15.103:0/2174694502 >> v2:172.21.15.122:3300/0 conn(0x7f33bc13f490 0x7f33bc13f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.232+0000 7f33c251d640 1 -- 172.21.15.103:0/2174694502 >> 172.21.15.103:0/2174694502 conn(0x7f33bc0b84f0 msgr2=0x7f33bc0b8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7f7e9d412640 1 --2- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e980a9e60 0x7f7e98122d70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800c430 tx=0x7f7e8800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff298ff9640 1 -- 172.21.15.103:0/366360000 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290018070 con 0x7ff294034700 2024-08-19T13:14:59.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff298ff9640 1 -- 172.21.15.103:0/366360000 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2900072f0 con 0x7ff294034700 2024-08-19T13:14:59.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7f7e8e7fc640 1 -- 172.21.15.103:0/4266759240 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88011790 con 0x7f7e980a9e60 2024-08-19T13:14:59.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7fafb0325640 1 -- start start 2024-08-19T13:14:59.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7fafb0325640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fafa806f4c0 0x7fafa8140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7fafb0325640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafa806f8a0 0x7fafa81dfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7fafb0325640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafa8144100 0x7fafa8149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff298ff9640 1 -- 172.21.15.103:0/366360000 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900047e0 con 0x7ff294034700 2024-08-19T13:14:59.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7fafb0325640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fafa81522e0 con 0x7fafa806f8a0 2024-08-19T13:14:59.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.233+0000 7ff2a1034640 1 -- 172.21.15.103:0/366360000 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2940c4730 con 0x7ff294034700 2024-08-19T13:14:59.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7ff2a1034640 1 -- 172.21.15.103:0/366360000 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2940c5940 con 0x7ff294034700 2024-08-19T13:14:59.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7f7e9ec15640 1 -- 172.21.15.103:0/4266759240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e98122080 con 0x7f7e980a9e60 2024-08-19T13:14:59.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7f7e9ec15640 1 -- 172.21.15.103:0/4266759240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e981e5260 con 0x7f7e980a9e60 2024-08-19T13:14:59.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7f215fb13640 1 Processor -- start 2024-08-19T13:14:59.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7f215fb13640 1 -- start start 2024-08-19T13:14:59.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7f7e8e7fc640 1 -- 172.21.15.103:0/4266759240 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e88017070 con 0x7f7e980a9e60 2024-08-19T13:14:59.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7f7e8e7fc640 1 -- 172.21.15.103:0/4266759240 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88011b70 con 0x7f7e980a9e60 2024-08-19T13:14:59.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7f33c251d640 1 -- 172.21.15.103:0/2174694502 shutdown_connections 2024-08-19T13:14:59.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7fafae8c2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafa8144100 0x7fafa8149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7fafae8c2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafa8144100 0x7fafa8149940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.103:48646/0 (socket says 172.21.15.103:48646) 2024-08-19T13:14:59.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7fafae8c2640 1 -- 172.21.15.103:0/1241242691 learned_addr learned my addr 172.21.15.103:0/1241242691 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7fafb0325640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8152160 con 0x7fafa8144100 2024-08-19T13:14:59.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7fafb0325640 1 -- 172.21.15.103:0/1241242691 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fafa8152460 con 0x7fafa806f4c0 2024-08-19T13:14:59.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f336c294640 1 Processor -- start 2024-08-19T13:14:59.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7ff2a1034640 1 -- 172.21.15.103:0/366360000 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff2940c4200 con 0x7ff294034700 2024-08-19T13:14:59.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7ff08df2c640 1 -- 172.21.15.103:0/234879209 shutdown_connections 2024-08-19T13:14:59.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f33c251d640 1 -- 172.21.15.103:0/2174694502 wait complete. 2024-08-19T13:14:59.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f7e9ec15640 1 -- 172.21.15.103:0/4266759240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7e80000f80 con 0x7f7e980a9e60 2024-08-19T13:14:59.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7ff08df2c640 1 -- 172.21.15.103:0/234879209 wait complete. 2024-08-19T13:14:59.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f682d5c6640 1 -- 172.21.15.103:0/2786204622 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f682813cb90 msgr2=0x7f68281d8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f682d5c6640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f682813cb90 0x7f68281d8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f215fb13640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2158072260 0x7f215812e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f682d5c6640 1 -- 172.21.15.103:0/2786204622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6828071740 msgr2=0x7f682813ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f682d5c6640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6828071740 0x7f682813ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f682d5c6640 1 -- 172.21.15.103:0/2786204622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682813b9e0 con 0x7f6828073700 2024-08-19T13:14:59.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f682ddc7640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6828071740 0x7f682813ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T13:14:59.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f215fb13640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21580ae7d0 0x7f215812efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f215fb13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f215813b4d0 0x7f215812f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f215fb13640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2158073940 con 0x7f215813b4d0 2024-08-19T13:14:59.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f215d8af640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2158072260 0x7f215812e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f215fb13640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f21580737c0 con 0x7f21580ae7d0 2024-08-19T13:14:59.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f215fb13640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f2158072be0 con 0x7f2158072260 2024-08-19T13:14:59.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.234+0000 7fafae0c1640 1 --2- 172.21.15.103:0/1241242691 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fafa806f4c0 0x7fafa8140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7fafad8c0640 1 --2- 172.21.15.103:0/1241242691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafa806f8a0 0x7fafa81dfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f336c294640 1 -- start start 2024-08-19T13:14:59.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f682e5c8640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f682813cb90 0x7f68281d8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T13:14:59.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f215d8af640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2158072260 0x7f215812e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48608/0 (socket says 172.21.15.103:48608) 2024-08-19T13:14:59.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7fafae8c2640 1 -- 172.21.15.103:0/1241242691 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fafa806f4c0 msgr2=0x7fafa8140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7fafae8c2640 1 --2- 172.21.15.103:0/1241242691 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fafa806f4c0 0x7fafa8140bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7fafae8c2640 1 -- 172.21.15.103:0/1241242691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafa806f8a0 msgr2=0x7fafa81dfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7fafae8c2640 1 --2- 172.21.15.103:0/1241242691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafa806f8a0 0x7fafa81dfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7fafae8c2640 1 -- 172.21.15.103:0/1241242691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa80a94c0 con 0x7fafa8144100 2024-08-19T13:14:59.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.235+0000 7f215d8af640 1 -- 172.21.15.103:0/4253521246 learned_addr learned my addr 172.21.15.103:0/4253521246 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.236+0000 7f215d0ae640 1 --2- 172.21.15.103:0/4253521246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21580ae7d0 0x7f215812efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.236+0000 7f215d8af640 1 -- 172.21.15.103:0/4253521246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21580ae7d0 msgr2=0x7f215812efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.236+0000 7f215d8af640 1 --2- 172.21.15.103:0/4253521246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21580ae7d0 0x7f215812efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.236+0000 7f215d8af640 1 -- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f215813b4d0 msgr2=0x7f215812f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.236+0000 7f215d8af640 1 --2- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f215813b4d0 0x7f215812f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.236+0000 7f215d8af640 1 -- 172.21.15.103:0/4253521246 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215812f500 con 0x7f2158072260 2024-08-19T13:14:59.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.236+0000 7f215d0ae640 1 --2- 172.21.15.103:0/4253521246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21580ae7d0 0x7f215812efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T13:14:59.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.236+0000 7f215d8af640 1 --2- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2158072260 0x7f215812e790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2148015cd0 tx=0x7f2148013560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.236+0000 7f682d5c6640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6828073700 0x7f682813b4a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f682400ec00 tx=0x7f682400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.236+0000 7fafae8c2640 1 --2- 172.21.15.103:0/1241242691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafa8144100 0x7fafa8149940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fafa000c730 tx=0x7fafa000cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.236+0000 7f21567fc640 1 -- 172.21.15.103:0/4253521246 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214801d020 con 0x7f2158072260 2024-08-19T13:14:59.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.236+0000 7f33c251d640 1 Processor -- start 2024-08-19T13:14:59.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.236+0000 7f215fb13640 1 -- 172.21.15.103:0/4253521246 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21581313f0 con 0x7f2158072260 2024-08-19T13:14:59.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.236+0000 7f681e7fc640 1 -- 172.21.15.103:0/2786204622 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824017070 con 0x7f6828073700 2024-08-19T13:14:59.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7ff08df2c640 1 Processor -- start 2024-08-19T13:14:59.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7f681e7fc640 1 -- 172.21.15.103:0/2786204622 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f682400ee10 con 0x7f6828073700 2024-08-19T13:14:59.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7f683002b640 1 -- 172.21.15.103:0/2786204622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682813e610 con 0x7f6828073700 2024-08-19T13:14:59.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7f215fb13640 1 -- 172.21.15.103:0/4253521246 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f215813aeb0 con 0x7f2158072260 2024-08-19T13:14:59.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7f683002b640 1 -- 172.21.15.103:0/2786204622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68281d88b0 con 0x7f6828073700 2024-08-19T13:14:59.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7f21567fc640 1 -- 172.21.15.103:0/4253521246 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2148019630 con 0x7f2158072260 2024-08-19T13:14:59.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7f21567fc640 1 -- 172.21.15.103:0/4253521246 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148012030 con 0x7f2158072260 2024-08-19T13:14:59.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f7e8e7fc640 1 -- 172.21.15.103:0/4266759240 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e88011d10 con 0x7f7e980a9e60 2024-08-19T13:14:59.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7f33c251d640 1 -- start start 2024-08-19T13:14:59.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f33c251d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33bc0aef70 0x7f33bc0a9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f33c251d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33bc13f490 0x7f33bc1d7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f681e7fc640 1 -- 172.21.15.103:0/2786204622 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824013470 con 0x7f6828073700 2024-08-19T13:14:59.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7f336c294640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f335c02bfe0 0x7f335c02e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7f336a030640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f335c02bfe0 0x7f335c02e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7f336a030640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f335c02bfe0 0x7f335c02e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49808/0 (socket says 172.21.15.103:49808) 2024-08-19T13:14:59.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7f336a030640 1 -- 172.21.15.103:0/3454398435 learned_addr learned my addr 172.21.15.103:0/3454398435 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7f336c294640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c033990 0x7f335c0c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7f336c294640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f335c036ea0 0x7f335c0c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f336c294640 1 -- 172.21.15.103:0/3454398435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f335c006620 con 0x7f335c02bfe0 2024-08-19T13:14:59.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f336c294640 1 -- 172.21.15.103:0/3454398435 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f335c0374c0 con 0x7f335c033990 2024-08-19T13:14:59.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f336c294640 1 -- 172.21.15.103:0/3454398435 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f335c0067a0 con 0x7f335c036ea0 2024-08-19T13:14:59.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f336982f640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c033990 0x7f335c0c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f33c251d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33bc14ae90 0x7f33bc13e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7faf9effd640 1 -- 172.21.15.103:0/1241242691 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0003b70 con 0x7fafa8144100 2024-08-19T13:14:59.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f215fb13640 1 -- 172.21.15.103:0/4253521246 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2138000f80 con 0x7f2158072260 2024-08-19T13:14:59.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f33c251d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc0afce0 con 0x7f33bc13f490 2024-08-19T13:14:59.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f33c251d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc13fde0 con 0x7f33bc0aef70 2024-08-19T13:14:59.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f33c251d640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f33bc0afe60 con 0x7f33bc14ae90 2024-08-19T13:14:59.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7fafb0325640 1 -- 172.21.15.103:0/1241242691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa814a0a0 con 0x7fafa8144100 2024-08-19T13:14:59.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7fafb0325640 1 -- 172.21.15.103:0/1241242691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa814a5e0 con 0x7fafa8144100 2024-08-19T13:14:59.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f336a831640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f335c036ea0 0x7f335c0c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f336a030640 1 -- 172.21.15.103:0/3454398435 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f335c036ea0 msgr2=0x7f335c0c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f336a030640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f335c036ea0 0x7f335c0c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7ff08df2c640 1 -- start start 2024-08-19T13:14:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7ff08df2c640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0880b14c0 0x7ff0880726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7ff08df2c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0880b4fa0 0x7ff088071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7ff08df2c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff088142330 0x7ff08806eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7ff08df2c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff08814cb60 con 0x7ff088142330 2024-08-19T13:14:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7ff08df2c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff08814cce0 con 0x7ff0880b4fa0 2024-08-19T13:14:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7ff08df2c640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff08814ce60 con 0x7ff0880b14c0 2024-08-19T13:14:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7ff0877fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0880b14c0 0x7ff0880726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7ff087fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff088142330 0x7ff08806eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7ff0877fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0880b14c0 0x7ff0880726b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48630/0 (socket says 172.21.15.103:48630) 2024-08-19T13:14:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7ff0877fe640 1 -- 172.21.15.103:0/1798101650 learned_addr learned my addr 172.21.15.103:0/1798101650 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7ff0877fe640 1 -- 172.21.15.103:0/1798101650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0880b4fa0 msgr2=0x7ff088071760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T13:14:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7ff0877fe640 1 --2- 172.21.15.103:0/1798101650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0880b4fa0 0x7ff088071760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7ff0877fe640 1 -- 172.21.15.103:0/1798101650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff088142330 msgr2=0x7ff08806eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f336a030640 1 -- 172.21.15.103:0/3454398435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c033990 msgr2=0x7f335c0c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f336a030640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c033990 0x7f335c0c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f336a030640 1 -- 172.21.15.103:0/3454398435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335c02fe40 con 0x7f335c02bfe0 2024-08-19T13:14:59.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f336a831640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f335c036ea0 0x7f335c0c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T13:14:59.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f336a030640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f335c02bfe0 0x7f335c02e660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3360009990 tx=0x7f33600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f336982f640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c033990 0x7f335c0c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T13:14:59.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7f680bfff640 1 -- 172.21.15.103:0/2786204622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f68280af880 con 0x7f6828073700 2024-08-19T13:14:59.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.237+0000 7ff298ff9640 1 -- 172.21.15.103:0/366360000 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff29001b070 con 0x7ff294034700 2024-08-19T13:14:59.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.238+0000 7ff298ff9640 1 --2- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff28006be30 0x7ff28006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f7e8e7fc640 1 --2- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7e7c06bf30 0x7f7e7c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7ff298ff9640 1 -- 172.21.15.103:0/366360000 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7ff29008a420 con 0x7ff294034700 2024-08-19T13:14:59.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f7e8e7fc640 1 -- 172.21.15.103:0/4266759240 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f7e8808b490 con 0x7f7e980a9e60 2024-08-19T13:14:59.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f335affd640 1 -- 172.21.15.103:0/3454398435 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360037040 con 0x7f335c02bfe0 2024-08-19T13:14:59.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f7e8e7fc640 1 --2- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:6800/4004268210,v1:172.21.15.103:6801/4004268210] conn(0x7f7e7c072370 0x7f7e7c0747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f335affd640 1 -- 172.21.15.103:0/3454398435 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3360042070 con 0x7f335c02bfe0 2024-08-19T13:14:59.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f336c294640 1 -- 172.21.15.103:0/3454398435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335c030120 con 0x7f335c02bfe0 2024-08-19T13:14:59.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.240+0000 7f7e8e7fc640 1 -- 172.21.15.103:0/4266759240 --> [v2:172.21.15.103:6800/4004268210,v1:172.21.15.103:6801/4004268210] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7e88014e70 con 0x7f7e7c072370 2024-08-19T13:14:59.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.240+0000 7f7e8e7fc640 1 -- 172.21.15.103:0/4266759240 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7e8808b880 con 0x7f7e980a9e60 2024-08-19T13:14:59.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.240+0000 7f7e9e414640 1 --2- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:6800/4004268210,v1:172.21.15.103:6801/4004268210] conn(0x7f7e7c072370 0x7f7e7c0747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f33c0aba640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33bc14ae90 0x7f33bc13e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f33c0aba640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33bc14ae90 0x7f33bc13e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48632/0 (socket says 172.21.15.103:48632) 2024-08-19T13:14:59.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f33c0aba640 1 -- 172.21.15.103:0/1176431895 learned_addr learned my addr 172.21.15.103:0/1176431895 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f33bbfff640 1 --2- 172.21.15.103:0/1176431895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33bc0aef70 0x7f33bc0a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f33c0aba640 1 -- 172.21.15.103:0/1176431895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33bc0aef70 msgr2=0x7f33bc0a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f33c0aba640 1 --2- 172.21.15.103:0/1176431895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33bc0aef70 0x7f33bc0a9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f33c0aba640 1 -- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33bc13f490 msgr2=0x7f33bc1d7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f33c0aba640 1 --2- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33bc13f490 0x7f33bc1d7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7f33c0aba640 1 -- 172.21.15.103:0/1176431895 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc0ab750 con 0x7f33bc14ae90 2024-08-19T13:14:59.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.240+0000 7f33bbfff640 1 --2- 172.21.15.103:0/1176431895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33bc0aef70 0x7f33bc0a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T13:14:59.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.240+0000 7f33c0aba640 1 --2- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33bc14ae90 0x7f33bc13e480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009eb0 tx=0x7f33b4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7ff298ff9640 1 --2- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:6824/855529591,v1:172.21.15.103:6825/855529591] conn(0x7ff280072270 0x7ff2800746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7ff298ff9640 1 -- 172.21.15.103:0/366360000 --> [v2:172.21.15.103:6824/855529591,v1:172.21.15.103:6825/855529591] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff280074d80 con 0x7ff280072270 2024-08-19T13:14:59.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7ff298ff9640 1 -- 172.21.15.103:0/366360000 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7ff29008a810 con 0x7ff294034700 2024-08-19T13:14:59.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.240+0000 7f21567fc640 1 -- 172.21.15.103:0/4253521246 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2148020020 con 0x7f2158072260 2024-08-19T13:14:59.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.240+0000 7f7e9dc13640 1 --2- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7e7c06bf30 0x7f7e7c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7ff29bfff640 1 --2- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff28006be30 0x7ff28006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.240+0000 7f33b8ff9640 1 -- 172.21.15.103:0/1176431895 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4004450 con 0x7f33bc14ae90 2024-08-19T13:14:59.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.240+0000 7f7e9e414640 1 --2- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:6800/4004268210,v1:172.21.15.103:6801/4004268210] conn(0x7f7e7c072370 0x7f7e7c0747b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.240+0000 7f33b8ff9640 1 -- 172.21.15.103:0/1176431895 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b40045f0 con 0x7f33bc14ae90 2024-08-19T13:14:59.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.240+0000 7f33b8ff9640 1 -- 172.21.15.103:0/1176431895 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b403b410 con 0x7f33bc14ae90 2024-08-19T13:14:59.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7faf9effd640 1 -- 172.21.15.103:0/1241242691 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa0003d10 con 0x7fafa8144100 2024-08-19T13:14:59.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.239+0000 7faf9effd640 1 -- 172.21.15.103:0/1241242691 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0005310 con 0x7fafa8144100 2024-08-19T13:14:59.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7f21567fc640 1 --2- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f214406bf00 0x7f214406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7ff29bfff640 1 --2- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff28006be30 0x7ff28006e2f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff288034000 tx=0x7ff28802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7ff2a0833640 1 --2- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:6824/855529591,v1:172.21.15.103:6825/855529591] conn(0x7ff280072270 0x7ff2800746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7f7e9dc13640 1 --2- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7e7c06bf30 0x7f7e7c06e3f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7e94034000 tx=0x7f7e9402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.242+0000 7f7e8e7fc640 1 -- 172.21.15.103:0/4266759240 <== osd.2 v2:172.21.15.103:6800/4004268210 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e88014e70 con 0x7f7e7c072370 2024-08-19T13:14:59.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.242+0000 7ff0877fe640 1 --2- 172.21.15.103:0/1798101650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff088142330 0x7ff08806eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.242+0000 7ff0877fe640 1 -- 172.21.15.103:0/1798101650 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff088147570 con 0x7ff0880b14c0 2024-08-19T13:14:59.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.242+0000 7ff0877fe640 1 --2- 172.21.15.103:0/1798101650 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0880b14c0 0x7ff0880726b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff07802a840 tx=0x7ff0780046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.242+0000 7ff087fff640 1 --2- 172.21.15.103:0/1798101650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff088142330 0x7ff08806eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T13:14:59.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7f33c251d640 1 -- 172.21.15.103:0/1176431895 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33bc0ab9c0 con 0x7f33bc14ae90 2024-08-19T13:14:59.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7f33c251d640 1 -- 172.21.15.103:0/1176431895 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f33bc0a9ec0 con 0x7f33bc14ae90 2024-08-19T13:14:59.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.242+0000 7fafb0325640 1 -- 172.21.15.103:0/1241242691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf90000f80 con 0x7fafa8144100 2024-08-19T13:14:59.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7f681e7fc640 1 -- 172.21.15.103:0/2786204622 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f682401a070 con 0x7f6828073700 2024-08-19T13:14:59.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7f681e7fc640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f680c06be60 0x7f680c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7f681e7fc640 1 -- 172.21.15.103:0/2786204622 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f682408b040 con 0x7f6828073700 2024-08-19T13:14:59.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.242+0000 7f681e7fc640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.122:6808/2033312647,v1:172.21.15.122:6809/2033312647] conn(0x7f680c0722a0 0x7f680c0746e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7f21567fc640 1 -- 172.21.15.103:0/4253521246 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f2148093e10 con 0x7f2158072260 2024-08-19T13:14:59.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7f21567fc640 1 --2- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:6808/1798955708,v1:172.21.15.103:6809/1798955708] conn(0x7f2144072340 0x7f2144074780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7f21567fc640 1 -- 172.21.15.103:0/4253521246 --> [v2:172.21.15.103:6808/1798955708,v1:172.21.15.103:6809/1798955708] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f214800dee0 con 0x7f2144072340 2024-08-19T13:14:59.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7f21567fc640 1 -- 172.21.15.103:0/4253521246 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f2148094200 con 0x7f2158072260 2024-08-19T13:14:59.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.242+0000 7f215e0b0640 1 --2- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:6808/1798955708,v1:172.21.15.103:6809/1798955708] conn(0x7f2144072340 0x7f2144074780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.242+0000 7f681e7fc640 1 -- 172.21.15.103:0/2786204622 --> [v2:172.21.15.122:6808/2033312647,v1:172.21.15.122:6809/2033312647] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6824007e50 con 0x7f680c0722a0 2024-08-19T13:14:59.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.242+0000 7f681e7fc640 1 -- 172.21.15.103:0/2786204622 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f682408b430 con 0x7f6828073700 2024-08-19T13:14:59.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.242+0000 7f682e5c8640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.122:6808/2033312647,v1:172.21.15.122:6809/2033312647] conn(0x7f680c0722a0 0x7f680c0746e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.242+0000 7faf9effd640 1 -- 172.21.15.103:0/1241242691 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fafa00054b0 con 0x7fafa8144100 2024-08-19T13:14:59.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.242+0000 7f682ddc7640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f680c06be60 0x7f680c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.242+0000 7f33a27fc640 1 -- 172.21.15.103:0/1176431895 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f33bc0a9ca0 con 0x7f33bc14ae90 2024-08-19T13:14:59.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.243+0000 7faf9effd640 1 --2- 172.21.15.103:0/1241242691 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7faf8c06bf00 0x7faf8c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7f336c294640 1 -- 172.21.15.103:0/3454398435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f335c02dda0 con 0x7f335c02bfe0 2024-08-19T13:14:59.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.241+0000 7f335affd640 1 -- 172.21.15.103:0/3454398435 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336003d440 con 0x7f335c02bfe0 2024-08-19T13:14:59.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.243+0000 7f215e0b0640 1 --2- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:6808/1798955708,v1:172.21.15.103:6809/1798955708] conn(0x7f2144072340 0x7f2144074780 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.243+0000 7faf9effd640 1 -- 172.21.15.103:0/1241242691 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7fafa008ab30 con 0x7fafa8144100 2024-08-19T13:14:59.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.243+0000 7f682ddc7640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f680c06be60 0x7f680c06e320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f681800c940 tx=0x7f6818008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.243+0000 7f336c294640 1 -- 172.21.15.103:0/3454398435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f334c000f80 con 0x7f335c02bfe0 2024-08-19T13:14:59.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.243+0000 7faf9effd640 1 --2- 172.21.15.103:0/1241242691 >> [v2:172.21.15.122:6800/634629443,v1:172.21.15.122:6801/634629443] conn(0x7faf8c072340 0x7faf8c074780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.243+0000 7ff2a0833640 1 --2- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:6824/855529591,v1:172.21.15.103:6825/855529591] conn(0x7ff280072270 0x7ff2800746b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.243+0000 7ff06ffff640 1 -- 172.21.15.103:0/1798101650 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078041070 con 0x7ff0880b14c0 2024-08-19T13:14:59.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.243+0000 7f682e5c8640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.122:6808/2033312647,v1:172.21.15.122:6809/2033312647] conn(0x7f680c0722a0 0x7f680c0746e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.243+0000 7ff06ffff640 1 -- 172.21.15.103:0/1798101650 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff078032e50 con 0x7ff0880b14c0 2024-08-19T13:14:59.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.244+0000 7faf9effd640 1 -- 172.21.15.103:0/1241242691 --> [v2:172.21.15.122:6800/634629443,v1:172.21.15.122:6801/634629443] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fafa0014e50 con 0x7faf8c072340 2024-08-19T13:14:59.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.244+0000 7faf9effd640 1 -- 172.21.15.103:0/1241242691 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fafa0055210 con 0x7fafa8144100 2024-08-19T13:14:59.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.244+0000 7f215d0ae640 1 --2- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f214406bf00 0x7f214406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.244+0000 7f21567fc640 1 -- 172.21.15.103:0/4253521246 <== osd.0 v2:172.21.15.103:6808/1798955708 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f214800dee0 con 0x7f2144072340 2024-08-19T13:14:59.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.244+0000 7ff06ffff640 1 -- 172.21.15.103:0/1798101650 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078037040 con 0x7ff0880b14c0 2024-08-19T13:14:59.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.244+0000 7ff08df2c640 1 -- 172.21.15.103:0/1798101650 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff088147850 con 0x7ff0880b14c0 2024-08-19T13:14:59.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.244+0000 7ff08df2c640 1 -- 172.21.15.103:0/1798101650 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff088146920 con 0x7ff0880b14c0 2024-08-19T13:14:59.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.244+0000 7f33b8ff9640 1 -- 172.21.15.103:0/1176431895 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33b4007790 con 0x7f33bc14ae90 2024-08-19T13:14:59.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.244+0000 7ff298ff9640 1 -- 172.21.15.103:0/366360000 <== osd.1 v2:172.21.15.103:6824/855529591 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff280074d80 con 0x7ff280072270 2024-08-19T13:14:59.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.244+0000 7fafad8c0640 1 --2- 172.21.15.103:0/1241242691 >> [v2:172.21.15.122:6800/634629443,v1:172.21.15.122:6801/634629443] conn(0x7faf8c072340 0x7faf8c074780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7fafae0c1640 1 --2- 172.21.15.103:0/1241242691 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7faf8c06bf00 0x7faf8c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7ff08df2c640 1 -- 172.21.15.103:0/1798101650 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff088072290 con 0x7ff0880b14c0 2024-08-19T13:14:59.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7f681e7fc640 1 -- 172.21.15.103:0/2786204622 <== osd.6 v2:172.21.15.122:6808/2033312647 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6824007e50 con 0x7f680c0722a0 2024-08-19T13:14:59.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7f33b8ff9640 1 --2- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f33a806be30 0x7f33a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7f33b8ff9640 1 -- 172.21.15.103:0/1176431895 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f33b40abd00 con 0x7f33bc14ae90 2024-08-19T13:14:59.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7f33b8ff9640 1 --2- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:6816/264232662,v1:172.21.15.103:6817/264232662] conn(0x7f33a8072270 0x7f33a80746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7f33b8ff9640 1 -- 172.21.15.103:0/1176431895 --> [v2:172.21.15.103:6816/264232662,v1:172.21.15.103:6817/264232662] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f33a8074d80 con 0x7f33a8072270 2024-08-19T13:14:59.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7f33b8ff9640 1 -- 172.21.15.103:0/1176431895 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f33b40ae360 con 0x7f33bc14ae90 2024-08-19T13:14:59.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7f33bbfff640 1 --2- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f33a806be30 0x7f33a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7f33bb7fe640 1 --2- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:6816/264232662,v1:172.21.15.103:6817/264232662] conn(0x7f33a8072270 0x7f33a80746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.244+0000 7f335affd640 1 -- 172.21.15.103:0/3454398435 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f336003d660 con 0x7f335c02bfe0 2024-08-19T13:14:59.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7f335affd640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f334806bf00 0x7f334806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7f335affd640 1 -- 172.21.15.103:0/3454398435 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f33600b0060 con 0x7f335c02bfe0 2024-08-19T13:14:59.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7f335affd640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.122:6824/3566117434,v1:172.21.15.122:6825/3566117434] conn(0x7f3348072340 0x7f3348074780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7f335affd640 1 -- 172.21.15.103:0/3454398435 --> [v2:172.21.15.122:6824/3566117434,v1:172.21.15.122:6825/3566117434] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3360032440 con 0x7f3348072340 2024-08-19T13:14:59.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.245+0000 7f335affd640 1 -- 172.21.15.103:0/3454398435 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f33600ab3b0 con 0x7f335c02bfe0 2024-08-19T13:14:59.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.246+0000 7fafad8c0640 1 --2- 172.21.15.103:0/1241242691 >> [v2:172.21.15.122:6800/634629443,v1:172.21.15.122:6801/634629443] conn(0x7faf8c072340 0x7faf8c074780 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.246+0000 7f336a831640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.122:6824/3566117434,v1:172.21.15.122:6825/3566117434] conn(0x7f3348072340 0x7f3348074780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.246+0000 7fafae0c1640 1 --2- 172.21.15.103:0/1241242691 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7faf8c06bf00 0x7faf8c06e3c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fafa4034000 tx=0x7fafa402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.246+0000 7f336982f640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f334806bf00 0x7f334806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.246+0000 7f215d0ae640 1 --2- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f214406bf00 0x7f214406e3c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2158132c60 tx=0x7f2150009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.246+0000 7ff06ffff640 1 -- 172.21.15.103:0/1798101650 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff078044070 con 0x7ff0880b14c0 2024-08-19T13:14:59.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.246+0000 7faf9effd640 1 -- 172.21.15.103:0/1241242691 <== osd.4 v2:172.21.15.122:6800/634629443 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fafa0014e50 con 0x7faf8c072340 2024-08-19T13:14:59.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.247+0000 7f336982f640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f334806bf00 0x7f334806e3c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f335c02f7b0 tx=0x7f3354009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.247+0000 7f33bb7fe640 1 --2- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:6816/264232662,v1:172.21.15.103:6817/264232662] conn(0x7f33a8072270 0x7f33a80746b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.247+0000 7f336a831640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.122:6824/3566117434,v1:172.21.15.122:6825/3566117434] conn(0x7f3348072340 0x7f3348074780 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.247+0000 7f33bbfff640 1 --2- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f33a806be30 0x7f33a806e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f33ac007fd0 tx=0x7f33ac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.247+0000 7ff06ffff640 1 --2- 172.21.15.103:0/1798101650 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff07406bf00 0x7ff07406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.247+0000 7ff086ffd640 1 --2- 172.21.15.103:0/1798101650 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff07406bf00 0x7ff07406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.247+0000 7ff06ffff640 1 -- 172.21.15.103:0/1798101650 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7ff0780abc80 con 0x7ff0880b14c0 2024-08-19T13:14:59.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.248+0000 7f335affd640 1 -- 172.21.15.103:0/3454398435 <== osd.7 v2:172.21.15.122:6824/3566117434 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3360032440 con 0x7f3348072340 2024-08-19T13:14:59.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.248+0000 7f33b8ff9640 1 -- 172.21.15.103:0/1176431895 <== osd.3 v2:172.21.15.103:6816/264232662 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f33a8074d80 con 0x7f33a8072270 2024-08-19T13:14:59.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.248+0000 7ff06ffff640 1 --2- 172.21.15.103:0/1798101650 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x7ff074072340 0x7ff074074780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.248+0000 7ff06ffff640 1 -- 172.21.15.103:0/1798101650 --> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff078001d30 con 0x7ff074072340 2024-08-19T13:14:59.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.248+0000 7ff06ffff640 1 -- 172.21.15.103:0/1798101650 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7ff0780ac070 con 0x7ff0880b14c0 2024-08-19T13:14:59.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.248+0000 7ff086ffd640 1 --2- 172.21.15.103:0/1798101650 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff07406bf00 0x7ff07406e3c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff088141d70 tx=0x7ff07c006b80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.249+0000 7ff087fff640 1 --2- 172.21.15.103:0/1798101650 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x7ff074072340 0x7ff074074780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.249+0000 7ff087fff640 1 --2- 172.21.15.103:0/1798101650 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x7ff074072340 0x7ff074074780 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.250+0000 7ff06ffff640 1 -- 172.21.15.103:0/1798101650 <== osd.5 v2:172.21.15.122:6816/2112974081 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff078001d30 con 0x7ff074072340 2024-08-19T13:14:59.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.272+0000 7f7e9ec15640 1 -- 172.21.15.103:0/4266759240 --> [v2:172.21.15.103:6800/4004268210,v1:172.21.15.103:6801/4004268210] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7e80002bf0 con 0x7f7e7c072370 2024-08-19T13:14:59.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.273+0000 7f215fb13640 1 -- 172.21.15.103:0/4253521246 --> [v2:172.21.15.103:6808/1798955708,v1:172.21.15.103:6809/1798955708] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2138002bf0 con 0x7f2144072340 2024-08-19T13:14:59.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.273+0000 7f7e8e7fc640 1 -- 172.21.15.103:0/4266759240 <== osd.2 v2:172.21.15.103:6800/4004268210 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7e80002bf0 con 0x7f7e7c072370 2024-08-19T13:14:59.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.273+0000 7f21567fc640 1 -- 172.21.15.103:0/4253521246 <== osd.0 v2:172.21.15.103:6808/1798955708 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2138002bf0 con 0x7f2144072340 2024-08-19T13:14:59.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.274+0000 7f2137fff640 1 -- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:6808/1798955708,v1:172.21.15.103:6809/1798955708] conn(0x7f2144072340 msgr2=0x7f2144074780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.274+0000 7f2137fff640 1 --2- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:6808/1798955708,v1:172.21.15.103:6809/1798955708] conn(0x7f2144072340 0x7f2144074780 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.274+0000 7f2137fff640 1 -- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f214406bf00 msgr2=0x7f214406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.274+0000 7f2137fff640 1 --2- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f214406bf00 0x7f214406e3c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2158132c60 tx=0x7f2150009040 comp rx=0 tx=0).stop 2024-08-19T13:14:59.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.274+0000 7f2137fff640 1 -- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2158072260 msgr2=0x7f215812e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.274+0000 7f2137fff640 1 --2- 172.21.15.103:0/4253521246 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2158072260 0x7f215812e790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2148015cd0 tx=0x7f2148013560 comp rx=0 tx=0).stop 2024-08-19T13:14:59.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.274+0000 7f215e0b0640 1 -- 172.21.15.103:0/4253521246 reap_dead start 2024-08-19T13:14:59.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.274+0000 7f7e9ec15640 1 -- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:6800/4004268210,v1:172.21.15.103:6801/4004268210] conn(0x7f7e7c072370 msgr2=0x7f7e7c0747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.274+0000 7f7e9ec15640 1 --2- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:6800/4004268210,v1:172.21.15.103:6801/4004268210] conn(0x7f7e7c072370 0x7f7e7c0747b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.274+0000 7f7e9ec15640 1 -- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7e7c06bf30 msgr2=0x7f7e7c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.275+0000 7f2137fff640 1 -- 172.21.15.103:0/4253521246 shutdown_connections 2024-08-19T13:14:59.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.275+0000 7f2137fff640 1 -- 172.21.15.103:0/4253521246 >> 172.21.15.103:0/4253521246 conn(0x7f21580b3fb0 msgr2=0x7f2158072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.274+0000 7f7e9ec15640 1 --2- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7e7c06bf30 0x7f7e7c06e3f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7e94034000 tx=0x7f7e9402e000 comp rx=0 tx=0).stop 2024-08-19T13:14:59.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.274+0000 7f7e9ec15640 1 -- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e980a9e60 msgr2=0x7f7e98122d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.274+0000 7f7e9ec15640 1 --2- 172.21.15.103:0/4266759240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e980a9e60 0x7f7e98122d70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800c430 tx=0x7f7e8800c8f0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.275+0000 7f7e9e414640 1 -- 172.21.15.103:0/4266759240 reap_dead start 2024-08-19T13:14:59.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.275+0000 7f7e9ec15640 1 -- 172.21.15.103:0/4266759240 shutdown_connections 2024-08-19T13:14:59.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.275+0000 7f7e9ec15640 1 -- 172.21.15.103:0/4266759240 >> 172.21.15.103:0/4266759240 conn(0x7f7e9806b800 msgr2=0x7f7e9806e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.275+0000 7f680bfff640 1 -- 172.21.15.103:0/2786204622 --> [v2:172.21.15.122:6808/2033312647,v1:172.21.15.122:6809/2033312647] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f68280b2c30 con 0x7f680c0722a0 2024-08-19T13:14:59.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.275+0000 7f2137fff640 1 -- 172.21.15.103:0/4253521246 shutdown_connections 2024-08-19T13:14:59.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.276+0000 7f2137fff640 1 -- 172.21.15.103:0/4253521246 wait complete. 2024-08-19T13:14:59.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.276+0000 7f7e9ec15640 1 -- 172.21.15.103:0/4266759240 shutdown_connections 2024-08-19T13:14:59.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.276+0000 7f7e9ec15640 1 -- 172.21.15.103:0/4266759240 wait complete. 2024-08-19T13:14:59.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.276+0000 7f681e7fc640 1 -- 172.21.15.103:0/2786204622 <== osd.6 v2:172.21.15.122:6808/2033312647 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f68280b2c30 con 0x7f680c0722a0 2024-08-19T13:14:59.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.277+0000 7ff2a1034640 1 -- 172.21.15.103:0/366360000 --> [v2:172.21.15.103:6824/855529591,v1:172.21.15.103:6825/855529591] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff2940c7900 con 0x7ff280072270 2024-08-19T13:14:59.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.277+0000 7f680bfff640 1 -- 172.21.15.103:0/2786204622 >> [v2:172.21.15.122:6808/2033312647,v1:172.21.15.122:6809/2033312647] conn(0x7f680c0722a0 msgr2=0x7f680c0746e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.277+0000 7f680bfff640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.122:6808/2033312647,v1:172.21.15.122:6809/2033312647] conn(0x7f680c0722a0 0x7f680c0746e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.277+0000 7ff298ff9640 1 -- 172.21.15.103:0/366360000 <== osd.1 v2:172.21.15.103:6824/855529591 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff2940c7900 con 0x7ff280072270 2024-08-19T13:14:59.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.277+0000 7f33a27fc640 1 -- 172.21.15.103:0/1176431895 --> [v2:172.21.15.103:6816/264232662,v1:172.21.15.103:6817/264232662] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f33bc0aa880 con 0x7f33a8072270 2024-08-19T13:14:59.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.277+0000 7f680bfff640 1 -- 172.21.15.103:0/2786204622 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f680c06be60 msgr2=0x7f680c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.277+0000 7f680bfff640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f680c06be60 0x7f680c06e320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f681800c940 tx=0x7f6818008040 comp rx=0 tx=0).stop 2024-08-19T13:14:59.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.277+0000 7f680bfff640 1 -- 172.21.15.103:0/2786204622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6828073700 msgr2=0x7f682813b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.277+0000 7f680bfff640 1 --2- 172.21.15.103:0/2786204622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6828073700 0x7f682813b4a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f682400ec00 tx=0x7f682400c520 comp rx=0 tx=0).stop 2024-08-19T13:14:59.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.277+0000 7f682e5c8640 1 -- 172.21.15.103:0/2786204622 reap_dead start 2024-08-19T13:14:59.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.277+0000 7f33b8ff9640 1 -- 172.21.15.103:0/1176431895 <== osd.3 v2:172.21.15.103:6816/264232662 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f33bc0aa880 con 0x7f33a8072270 2024-08-19T13:14:59.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7f680bfff640 1 -- 172.21.15.103:0/2786204622 shutdown_connections 2024-08-19T13:14:59.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7ff2a1034640 1 -- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:6824/855529591,v1:172.21.15.103:6825/855529591] conn(0x7ff280072270 msgr2=0x7ff2800746b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7ff2a1034640 1 --2- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:6824/855529591,v1:172.21.15.103:6825/855529591] conn(0x7ff280072270 0x7ff2800746b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7f680bfff640 1 -- 172.21.15.103:0/2786204622 >> 172.21.15.103:0/2786204622 conn(0x7f68280b3fb0 msgr2=0x7f682806f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7ff2a1034640 1 -- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff28006be30 msgr2=0x7ff28006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7ff2a1034640 1 --2- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff28006be30 0x7ff28006e2f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff288034000 tx=0x7ff28802e000 comp rx=0 tx=0).stop 2024-08-19T13:14:59.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7ff2a1034640 1 -- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff294034700 msgr2=0x7ff2940c4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7ff2a1034640 1 --2- 172.21.15.103:0/366360000 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff294034700 0x7ff2940c4ec0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff29000c750 tx=0x7ff29000cc10 comp rx=0 tx=0).stop 2024-08-19T13:14:59.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7ff2a0833640 1 -- 172.21.15.103:0/366360000 reap_dead start 2024-08-19T13:14:59.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7ff2a1034640 1 -- 172.21.15.103:0/366360000 shutdown_connections 2024-08-19T13:14:59.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7ff2a1034640 1 -- 172.21.15.103:0/366360000 >> 172.21.15.103:0/366360000 conn(0x7ff294005e20 msgr2=0x7ff29402a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7f680bfff640 1 -- 172.21.15.103:0/2786204622 shutdown_connections 2024-08-19T13:14:59.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7f680bfff640 1 -- 172.21.15.103:0/2786204622 wait complete. 2024-08-19T13:14:59.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7ff2a1034640 1 -- 172.21.15.103:0/366360000 shutdown_connections 2024-08-19T13:14:59.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7f33a27fc640 1 -- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:6816/264232662,v1:172.21.15.103:6817/264232662] conn(0x7f33a8072270 msgr2=0x7f33a80746b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7f33a27fc640 1 --2- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:6816/264232662,v1:172.21.15.103:6817/264232662] conn(0x7f33a8072270 0x7f33a80746b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7f33a27fc640 1 -- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f33a806be30 msgr2=0x7f33a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7f33a27fc640 1 --2- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f33a806be30 0x7f33a806e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f33ac007fd0 tx=0x7f33ac009040 comp rx=0 tx=0).stop 2024-08-19T13:14:59.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7f33a27fc640 1 -- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33bc14ae90 msgr2=0x7f33bc13e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7f33a27fc640 1 --2- 172.21.15.103:0/1176431895 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33bc14ae90 0x7f33bc13e480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009eb0 tx=0x7f33b4004a40 comp rx=0 tx=0).stop 2024-08-19T13:14:59.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.278+0000 7f33c0aba640 1 -- 172.21.15.103:0/1176431895 reap_dead start 2024-08-19T13:14:59.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.279+0000 7ff2a1034640 1 -- 172.21.15.103:0/366360000 wait complete. 2024-08-19T13:14:59.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.279+0000 7f33a27fc640 1 -- 172.21.15.103:0/1176431895 shutdown_connections 2024-08-19T13:14:59.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.279+0000 7f33a27fc640 1 -- 172.21.15.103:0/1176431895 >> 172.21.15.103:0/1176431895 conn(0x7f33bc0b84f0 msgr2=0x7f33bc0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.279+0000 7fafb0325640 1 -- 172.21.15.103:0/1241242691 --> [v2:172.21.15.122:6800/634629443,v1:172.21.15.122:6801/634629443] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faf90002c70 con 0x7faf8c072340 2024-08-19T13:14:59.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.279+0000 7f336c294640 1 -- 172.21.15.103:0/3454398435 --> [v2:172.21.15.122:6824/3566117434,v1:172.21.15.122:6825/3566117434] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f334c002bf0 con 0x7f3348072340 2024-08-19T13:14:59.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.279+0000 7f33a27fc640 1 -- 172.21.15.103:0/1176431895 shutdown_connections 2024-08-19T13:14:59.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.279+0000 7f33a27fc640 1 -- 172.21.15.103:0/1176431895 wait complete. 2024-08-19T13:14:59.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.280+0000 7faf9effd640 1 -- 172.21.15.103:0/1241242691 <== osd.4 v2:172.21.15.122:6800/634629443 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faf90002c70 con 0x7faf8c072340 2024-08-19T13:14:59.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.280+0000 7f335affd640 1 -- 172.21.15.103:0/3454398435 <== osd.7 v2:172.21.15.122:6824/3566117434 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f334c002bf0 con 0x7f3348072340 2024-08-19T13:14:59.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.280+0000 7ff08df2c640 1 -- 172.21.15.103:0/1798101650 --> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff0880b0f80 con 0x7ff074072340 2024-08-19T13:14:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.280+0000 7f336c294640 1 -- 172.21.15.103:0/3454398435 >> [v2:172.21.15.122:6824/3566117434,v1:172.21.15.122:6825/3566117434] conn(0x7f3348072340 msgr2=0x7f3348074780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.280+0000 7f336c294640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.122:6824/3566117434,v1:172.21.15.122:6825/3566117434] conn(0x7f3348072340 0x7f3348074780 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.280+0000 7fafb0325640 1 -- 172.21.15.103:0/1241242691 >> [v2:172.21.15.122:6800/634629443,v1:172.21.15.122:6801/634629443] conn(0x7faf8c072340 msgr2=0x7faf8c074780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.280+0000 7fafb0325640 1 --2- 172.21.15.103:0/1241242691 >> [v2:172.21.15.122:6800/634629443,v1:172.21.15.122:6801/634629443] conn(0x7faf8c072340 0x7faf8c074780 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7fafb0325640 1 -- 172.21.15.103:0/1241242691 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7faf8c06bf00 msgr2=0x7faf8c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7fafb0325640 1 --2- 172.21.15.103:0/1241242691 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7faf8c06bf00 0x7faf8c06e3c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fafa4034000 tx=0x7fafa402e000 comp rx=0 tx=0).stop 2024-08-19T13:14:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7fafb0325640 1 -- 172.21.15.103:0/1241242691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafa8144100 msgr2=0x7fafa8149940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7fafb0325640 1 --2- 172.21.15.103:0/1241242691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafa8144100 0x7fafa8149940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fafa000c730 tx=0x7fafa000cbf0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7f336c294640 1 -- 172.21.15.103:0/3454398435 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f334806bf00 msgr2=0x7f334806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7f336c294640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f334806bf00 0x7f334806e3c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f335c02f7b0 tx=0x7f3354009040 comp rx=0 tx=0).stop 2024-08-19T13:14:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7f336c294640 1 -- 172.21.15.103:0/3454398435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f335c02bfe0 msgr2=0x7f335c02e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7f336c294640 1 --2- 172.21.15.103:0/3454398435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f335c02bfe0 0x7f335c02e660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3360009990 tx=0x7f33600046c0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7f336a831640 1 -- 172.21.15.103:0/3454398435 reap_dead start 2024-08-19T13:14:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7f336c294640 1 -- 172.21.15.103:0/3454398435 shutdown_connections 2024-08-19T13:14:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7f336c294640 1 -- 172.21.15.103:0/3454398435 >> 172.21.15.103:0/3454398435 conn(0x7f335c03a420 msgr2=0x7f335c0354e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7fafae8c2640 1 -- 172.21.15.103:0/1241242691 reap_dead start 2024-08-19T13:14:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7fafb0325640 1 -- 172.21.15.103:0/1241242691 shutdown_connections 2024-08-19T13:14:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7fafb0325640 1 -- 172.21.15.103:0/1241242691 >> 172.21.15.103:0/1241242691 conn(0x7fafa80b41c0 msgr2=0x7fafa80af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7ff06ffff640 1 -- 172.21.15.103:0/1798101650 <== osd.5 v2:172.21.15.122:6816/2112974081 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff0880b0f80 con 0x7ff074072340 2024-08-19T13:14:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.281+0000 7f336c294640 1 -- 172.21.15.103:0/3454398435 shutdown_connections 2024-08-19T13:14:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.282+0000 7fafb0325640 1 -- 172.21.15.103:0/1241242691 shutdown_connections 2024-08-19T13:14:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.282+0000 7f336c294640 1 -- 172.21.15.103:0/3454398435 wait complete. 2024-08-19T13:14:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.282+0000 7fafb0325640 1 -- 172.21.15.103:0/1241242691 wait complete. 2024-08-19T13:14:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.282+0000 7ff08df2c640 1 -- 172.21.15.103:0/1798101650 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x7ff074072340 msgr2=0x7ff074074780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.282+0000 7ff08df2c640 1 --2- 172.21.15.103:0/1798101650 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x7ff074072340 0x7ff074074780 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.282+0000 7ff08df2c640 1 -- 172.21.15.103:0/1798101650 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff07406bf00 msgr2=0x7ff07406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.282+0000 7ff08df2c640 1 --2- 172.21.15.103:0/1798101650 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff07406bf00 0x7ff07406e3c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff088141d70 tx=0x7ff07c006b80 comp rx=0 tx=0).stop 2024-08-19T13:14:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.282+0000 7ff08df2c640 1 -- 172.21.15.103:0/1798101650 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0880b14c0 msgr2=0x7ff0880726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.282+0000 7ff08df2c640 1 --2- 172.21.15.103:0/1798101650 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0880b14c0 0x7ff0880726b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff07802a840 tx=0x7ff0780046c0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.282+0000 7ff087fff640 1 -- 172.21.15.103:0/1798101650 reap_dead start 2024-08-19T13:14:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.283+0000 7ff08df2c640 1 -- 172.21.15.103:0/1798101650 shutdown_connections 2024-08-19T13:14:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.283+0000 7ff08df2c640 1 -- 172.21.15.103:0/1798101650 >> 172.21.15.103:0/1798101650 conn(0x7ff088073110 msgr2=0x7ff088075000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.283+0000 7ff08df2c640 1 -- 172.21.15.103:0/1798101650 shutdown_connections 2024-08-19T13:14:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.283+0000 7ff08df2c640 1 -- 172.21.15.103:0/1798101650 wait complete. 2024-08-19T13:14:59.345 INFO:teuthology.orchestra.run.smithi103.stdout:55834574852 2024-08-19T13:14:59.345 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-19T13:14:59.346 INFO:teuthology.orchestra.run.smithi103.stdout:55834574852 2024-08-19T13:14:59.346 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-19T13:14:59.347 INFO:teuthology.orchestra.run.smithi103.stdout:55834574853 2024-08-19T13:14:59.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-19T13:14:59.348 INFO:teuthology.orchestra.run.smithi103.stdout:55834574852 2024-08-19T13:14:59.348 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-08-19T13:14:59.349 INFO:teuthology.orchestra.run.smithi103.stdout:55834574852 2024-08-19T13:14:59.349 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-19T13:14:59.350 INFO:teuthology.orchestra.run.smithi103.stdout:55834574852 2024-08-19T13:14:59.350 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-19T13:14:59.351 INFO:teuthology.orchestra.run.smithi103.stdout:55834574852 2024-08-19T13:14:59.351 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-19T13:14:59.352 INFO:teuthology.orchestra.run.smithi103.stdout:55834574851 2024-08-19T13:14:59.352 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-08-19T13:14:59.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.551+0000 7feff4d08640 1 Processor -- start 2024-08-19T13:14:59.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.551+0000 7f8279c63640 1 Processor -- start 2024-08-19T13:14:59.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.551+0000 7feff4d08640 1 -- start start 2024-08-19T13:14:59.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.551+0000 7fad7bf76640 1 Processor -- start 2024-08-19T13:14:59.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7fad7bf76640 1 -- start start 2024-08-19T13:14:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7feff4d08640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feff00a9a90 0x7feff00aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7feff4d08640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7feff00ae450 0x7feff00ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7feff4d08640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feff00ab720 con 0x7feff00ac920 2024-08-19T13:14:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7feff4d08640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7feff00aafe0 con 0x7feff00ae450 2024-08-19T13:14:59.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.551+0000 7f8279c63640 1 -- start start 2024-08-19T13:14:59.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7f8279c63640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f826c0c0940 0x7f826c034490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7f8279c63640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f826c02afb0 0x7f826c0c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7f8279c63640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f826c02ab00 con 0x7f826c02b3d0 2024-08-19T13:14:59.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7f8279c63640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f826c02a480 con 0x7f826c02afb0 2024-08-19T13:14:59.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7f8279c63640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f826c0c39e0 con 0x7f826c0c0940 2024-08-19T13:14:59.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7feff4d08640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7feff00aba00 con 0x7feff00a9a90 2024-08-19T13:14:59.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fad7bf76640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fad7406d9e0 0x7fad7413b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fad7bf76640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad7413be40 con 0x7fad7413b9f0 2024-08-19T13:14:59.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fad7bf76640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad74070ac0 con 0x7fad74072260 2024-08-19T13:14:59.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fad7bf76640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fad7406f6a0 con 0x7fad7406d9e0 2024-08-19T13:14:59.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fad79511640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fad7413b9f0 0x7fad740703b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:51106/0 (socket says 172.21.15.103:51106) 2024-08-19T13:14:59.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fad79511640 1 -- 172.21.15.103:0/2487108182 learned_addr learned my addr 172.21.15.103:0/2487108182 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:14:59.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fad79d12640 1 --2- 172.21.15.103:0/2487108182 >> v2:172.21.15.122:3300/0 conn(0x7fad7406d9e0 0x7fad7413b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7f8273fff640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f826c02afb0 0x7f826c0c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7f8273fff640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f826c02afb0 0x7f826c0c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49844/0 (socket says 172.21.15.103:49844) 2024-08-19T13:14:59.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7f8273fff640 1 -- 172.21.15.103:0/856922849 learned_addr learned my addr 172.21.15.103:0/856922849 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7f8278c61640 1 --2- 172.21.15.103:0/856922849 >> v2:172.21.15.122:3300/0 conn(0x7f826c0c0940 0x7f826c034490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7f8273fff640 1 -- 172.21.15.103:0/856922849 >> v2:172.21.15.122:3300/0 conn(0x7f826c0c0940 msgr2=0x7f826c034490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7f8273fff640 1 --2- 172.21.15.103:0/856922849 >> v2:172.21.15.122:3300/0 conn(0x7f826c0c0940 0x7f826c034490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7f8273fff640 1 -- 172.21.15.103:0/856922849 >> v1:172.21.15.122:6789/0 conn(0x7f826c02b3d0 legacy=0x7f826c0c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fad78d10640 1 -- 172.21.15.103:0/2487108182 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2843575656 0 0) 0x7fad7413be40 con 0x7fad7413b9f0 2024-08-19T13:14:59.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7f8273fff640 1 -- 172.21.15.103:0/856922849 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c02c480 con 0x7f826c02afb0 2024-08-19T13:14:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fad78d10640 1 -- 172.21.15.103:0/2487108182 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad5c003520 con 0x7fad7413b9f0 2024-08-19T13:14:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7fefeffff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7feff00ac920 0x7feff00acd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.103:60012/0 (socket says 172.21.15.103:60012) 2024-08-19T13:14:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7fefeffff640 1 -- 172.21.15.103:0/307709623 learned_addr learned my addr 172.21.15.103:0/307709623 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:14:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7fefeeffd640 1 --2- 172.21.15.103:0/307709623 >> v2:172.21.15.103:3300/0 conn(0x7feff00ae450 0x7feff00ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.552+0000 7fefef7fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feff00a9a90 0x7feff00aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fefee7fc640 1 -- 172.21.15.103:0/307709623 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3456467355 0 0) 0x7feff00ab720 con 0x7feff00ac920 2024-08-19T13:14:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fefee7fc640 1 -- 172.21.15.103:0/307709623 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefd8003520 con 0x7feff00ac920 2024-08-19T13:14:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fefef7fe640 1 -- 172.21.15.103:0/307709623 >> v2:172.21.15.103:3300/0 conn(0x7feff00ae450 msgr2=0x7feff00ac3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fefef7fe640 1 --2- 172.21.15.103:0/307709623 >> v2:172.21.15.103:3300/0 conn(0x7feff00ae450 0x7feff00ac3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fefef7fe640 1 -- 172.21.15.103:0/307709623 >> v1:172.21.15.122:6789/0 conn(0x7feff00ac920 legacy=0x7feff00acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.553+0000 7fefef7fe640 1 -- 172.21.15.103:0/307709623 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff00abce0 con 0x7feff00a9a90 2024-08-19T13:14:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.554+0000 7f3229a2d640 1 Processor -- start 2024-08-19T13:14:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.554+0000 7f3229a2d640 1 -- start start 2024-08-19T13:14:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.554+0000 7fad78d10640 1 -- 172.21.15.103:0/2487108182 <== mon.0 v1:172.21.15.103:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1968343882 0 0) 0x7fad5c003520 con 0x7fad7413b9f0 2024-08-19T13:14:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.554+0000 7fad78d10640 1 -- 172.21.15.103:0/2487108182 >> v2:172.21.15.122:3300/0 conn(0x7fad7406d9e0 msgr2=0x7fad7413b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.554+0000 7fad78d10640 1 --2- 172.21.15.103:0/2487108182 >> v2:172.21.15.122:3300/0 conn(0x7fad7406d9e0 0x7fad7413b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.554+0000 7fad78d10640 1 -- 172.21.15.103:0/2487108182 >> v1:172.21.15.122:6789/0 conn(0x7fad74072260 legacy=0x7fad740726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.554+0000 7fad78d10640 1 -- 172.21.15.103:0/2487108182 --> v1:172.21.15.103:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad74071b60 con 0x7fad7413b9f0 2024-08-19T13:14:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.554+0000 7fefef7fe640 1 --2- 172.21.15.103:0/307709623 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feff00a9a90 0x7feff00aaaa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fefe00097a0 tx=0x7fefe002a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87260f960afc9988 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.554+0000 7fad78d10640 1 -- 172.21.15.103:0/2487108182 <== mon.0 v1:172.21.15.103:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3066137496 0 0) 0x7fad680015a0 con 0x7fad7413b9f0 2024-08-19T13:14:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fefee7fc640 1 -- 172.21.15.103:0/307709623 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0037070 con 0x7feff00a9a90 2024-08-19T13:14:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fefee7fc640 1 -- 172.21.15.103:0/307709623 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe002adb0 con 0x7feff00a9a90 2024-08-19T13:14:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fad78d10640 1 -- 172.21.15.103:0/2487108182 >> v1:172.21.15.103:6789/0 conn(0x7fad7413b9f0 legacy=0x7fad740703b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fad78d10640 1 --2- 172.21.15.103:0/2487108182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c003a50 0x7fad5c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fad78d10640 1 --2- 172.21.15.103:0/2487108182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad5c004800 0x7fad5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fad78d10640 1 --2- 172.21.15.103:0/2487108182 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad5c0051c0 0x7fad5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fad79d12640 1 --2- 172.21.15.103:0/2487108182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c003a50 0x7fad5c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fad78d10640 1 -- 172.21.15.103:0/2487108182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad7406f6a0 con 0x7fad5c004800 2024-08-19T13:14:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fad78d10640 1 -- 172.21.15.103:0/2487108182 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c00c160 con 0x7fad5c003a50 2024-08-19T13:14:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fad78d10640 1 -- 172.21.15.103:0/2487108182 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fad5c00c360 con 0x7fad5c0051c0 2024-08-19T13:14:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7f3229a2d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f321c0c0af0 0x7f321c0c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7f3228a2b640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f321c034f90 0x7f321c0c3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.103:60028/0 (socket says 172.21.15.103:60028) 2024-08-19T13:14:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7f3228a2b640 1 -- 172.21.15.103:0/4179237724 learned_addr learned my addr 172.21.15.103:0/4179237724 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:14:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7f322922c640 1 --2- 172.21.15.103:0/4179237724 >> v2:172.21.15.122:3300/0 conn(0x7f321c0c0af0 0x7f321c0c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7f3229a2d640 1 -- 172.21.15.103:0/4179237724 --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321c02c8e0 con 0x7f321c0c4610 2024-08-19T13:14:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7f3229a2d640 1 -- 172.21.15.103:0/4179237724 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321c02cbc0 con 0x7f321c034f90 2024-08-19T13:14:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fefee7fc640 1 -- 172.21.15.103:0/307709623 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0037070 con 0x7feff00a9a90 2024-08-19T13:14:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fad7a513640 1 --2- 172.21.15.103:0/2487108182 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad5c0051c0 0x7fad5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7f3229a2d640 1 -- 172.21.15.103:0/4179237724 --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f321c035370 con 0x7f321c0c0af0 2024-08-19T13:14:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7feff4d08640 1 -- 172.21.15.103:0/307709623 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feff00a9a90 msgr2=0x7feff00aaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7feff4d08640 1 --2- 172.21.15.103:0/307709623 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feff00a9a90 0x7feff00aaaa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fefe00097a0 tx=0x7fefe002a060 comp rx=0 tx=0).stop 2024-08-19T13:14:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fec0372c640 1 Processor -- start 2024-08-19T13:14:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fec0372c640 1 -- start start 2024-08-19T13:14:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7feff4d08640 1 -- 172.21.15.103:0/307709623 shutdown_connections 2024-08-19T13:14:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7feff4d08640 1 --2- 172.21.15.103:0/307709623 >> v2:172.21.15.103:3300/0 conn(0x7feff00ae450 0x7feff00ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7feff4d08640 1 --2- 172.21.15.103:0/307709623 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feff00a9a90 0x7feff00aaaa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7feff4d08640 1 -- 172.21.15.103:0/307709623 >> 172.21.15.103:0/307709623 conn(0x7feff00b0b80 msgr2=0x7feff006b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.554+0000 7f8278c61640 1 --2- 172.21.15.103:0/856922849 >> v2:172.21.15.122:3300/0 conn(0x7f826c0c0940 0x7f826c034490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T13:14:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.554+0000 7f8273fff640 1 --2- 172.21.15.103:0/856922849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f826c02afb0 0x7f826c0c6b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82740745e0 tx=0x7f82740a99e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c2aadf89b82530 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7f82737fe640 1 -- 172.21.15.103:0/856922849 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82740b2070 con 0x7f826c02afb0 2024-08-19T13:14:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7f82737fe640 1 -- 172.21.15.103:0/856922849 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82740ae960 con 0x7f826c02afb0 2024-08-19T13:14:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7f82737fe640 1 -- 172.21.15.103:0/856922849 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82740aedd0 con 0x7f826c02afb0 2024-08-19T13:14:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fcf32824640 1 Processor -- start 2024-08-19T13:14:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fcf32824640 1 -- start start 2024-08-19T13:14:59.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fc7b3fff640 1 Processor -- start 2024-08-19T13:14:59.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.555+0000 7fc7b3fff640 1 -- start start 2024-08-19T13:14:59.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7feff4d08640 1 -- 172.21.15.103:0/307709623 shutdown_connections 2024-08-19T13:14:59.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7feff4d08640 1 -- 172.21.15.103:0/307709623 wait complete. 2024-08-19T13:14:59.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b41462b0 0x7fc7b4073090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fc7b3fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc7b406f200 0x7fc7b4149c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fc7b3fff640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b406f690 con 0x7fc7b4070470 2024-08-19T13:14:59.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fc7b3fff640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc7b4070850 con 0x7fc7b406f200 2024-08-19T13:14:59.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fc7b3fff640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b406fe60 con 0x7fc7b41462b0 2024-08-19T13:14:59.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b2ffd640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fc7b4070470 0x7fc7b4145ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:51124/0 (socket says 172.21.15.103:51124) 2024-08-19T13:14:59.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b2ffd640 1 -- 172.21.15.103:0/1831650720 learned_addr learned my addr 172.21.15.103:0/1831650720 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:14:59.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b37fe640 1 --2- 172.21.15.103:0/1831650720 >> v2:172.21.15.122:3300/0 conn(0x7fc7b406f200 0x7fc7b4149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fec0372c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7febfc06f4e0 0x7febfc072c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fec0372c640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febfc06e950 con 0x7febfc072410 2024-08-19T13:14:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fec0372c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febfc0719a0 con 0x7febfc14f450 2024-08-19T13:14:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fec0372c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7febfc0c1740 con 0x7febfc06f4e0 2024-08-19T13:14:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fec00cc7640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7febfc06f4e0 0x7febfc072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fec00cc7640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7febfc06f4e0 0x7febfc072c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.103:48716/0 (socket says 172.21.15.103:48716) 2024-08-19T13:14:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fec00cc7640 1 -- 172.21.15.103:0/3476300621 learned_addr learned my addr 172.21.15.103:0/3476300621 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7f32237fe640 1 -- 172.21.15.103:0/4179237724 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4211379216 0 0) 0x7f321c02cbc0 con 0x7f321c034f90 2024-08-19T13:14:59.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7f32237fe640 1 -- 172.21.15.103:0/4179237724 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f320c003520 con 0x7f321c034f90 2024-08-19T13:14:59.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7f32237fe640 1 -- 172.21.15.103:0/4179237724 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2707397964 0 0) 0x7f321c02c8e0 con 0x7f321c0c4610 2024-08-19T13:14:59.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7f32237fe640 1 -- 172.21.15.103:0/4179237724 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f321c02cbc0 con 0x7f321c0c4610 2024-08-19T13:14:59.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7f322922c640 1 -- 172.21.15.103:0/4179237724 >> v1:172.21.15.122:6789/0 conn(0x7f321c034f90 legacy=0x7f321c0c3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7f322922c640 1 -- 172.21.15.103:0/4179237724 >> v1:172.21.15.103:6789/0 conn(0x7f321c0c4610 legacy=0x7f321c0c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7f322922c640 1 -- 172.21.15.103:0/4179237724 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c0c40f0 con 0x7f321c0c0af0 2024-08-19T13:14:59.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 -- 172.21.15.103:0/1831650720 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2009327625 0 0) 0x7fc7b406f690 con 0x7fc7b4070470 2024-08-19T13:14:59.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 -- 172.21.15.103:0/1831650720 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc79c003520 con 0x7fc7b4070470 2024-08-19T13:14:59.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fad7a513640 1 -- 172.21.15.103:0/2487108182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c003a50 msgr2=0x7fad5c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fad7a513640 1 --2- 172.21.15.103:0/2487108182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c003a50 0x7fad5c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fad7a513640 1 -- 172.21.15.103:0/2487108182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad5c004800 msgr2=0x7fad5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fad7a513640 1 --2- 172.21.15.103:0/2487108182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad5c004800 0x7fad5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fad7a513640 1 -- 172.21.15.103:0/2487108182 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c003520 con 0x7fad5c0051c0 2024-08-19T13:14:59.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.556+0000 7fad79d12640 1 --2- 172.21.15.103:0/2487108182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c003a50 0x7fad5c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:59.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fad7a513640 1 -- 172.21.15.103:0/2487108182 reap_dead start 2024-08-19T13:14:59.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b27fc640 1 --2- 172.21.15.103:0/1831650720 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b41462b0 0x7fc7b4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 -- 172.21.15.103:0/1831650720 <== mon.0 v1:172.21.15.103:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1160669005 0 0) 0x7fc79c003520 con 0x7fc7b4070470 2024-08-19T13:14:59.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 -- 172.21.15.103:0/1831650720 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b41462b0 msgr2=0x7fc7b4073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 --2- 172.21.15.103:0/1831650720 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b41462b0 0x7fc7b4073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 -- 172.21.15.103:0/1831650720 >> v2:172.21.15.122:3300/0 conn(0x7fc7b406f200 msgr2=0x7fc7b4149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 --2- 172.21.15.103:0/1831650720 >> v2:172.21.15.122:3300/0 conn(0x7fc7b406f200 0x7fc7b4149c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 -- 172.21.15.103:0/1831650720 --> v1:172.21.15.103:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b414a1c0 con 0x7fc7b4070470 2024-08-19T13:14:59.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 -- 172.21.15.103:0/1831650720 <== mon.0 v1:172.21.15.103:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3066137496 0 0) 0x7fc7a4001650 con 0x7fc7b4070470 2024-08-19T13:14:59.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 -- 172.21.15.103:0/1831650720 >> v1:172.21.15.103:6789/0 conn(0x7fc7b4070470 legacy=0x7fc7b4145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 --2- 172.21.15.103:0/1831650720 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b41462b0 0x7fc79c003a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 --2- 172.21.15.103:0/1831650720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc79c003fc0 0x7fc79c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 --2- 172.21.15.103:0/1831650720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc79c004d40 0x7fc79c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 -- 172.21.15.103:0/1831650720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4070850 con 0x7fc79c003fc0 2024-08-19T13:14:59.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 -- 172.21.15.103:0/1831650720 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c00ba70 con 0x7fc79c004d40 2024-08-19T13:14:59.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fc7b1ffb640 1 -- 172.21.15.103:0/1831650720 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc79c00bc70 con 0x7fc7b41462b0 2024-08-19T13:14:59.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fec00cc7640 1 -- 172.21.15.103:0/3476300621 >> v1:172.21.15.122:6789/0 conn(0x7febfc14f450 legacy=0x7febfc14f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fec00cc7640 1 -- 172.21.15.103:0/3476300621 >> v1:172.21.15.103:6789/0 conn(0x7febfc072410 legacy=0x7febfc06fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.557+0000 7fec00cc7640 1 -- 172.21.15.103:0/3476300621 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febfc06f970 con 0x7febfc06f4e0 2024-08-19T13:14:59.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7feff4d08640 1 Processor -- start 2024-08-19T13:14:59.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7fc7b27fc640 1 --2- 172.21.15.103:0/1831650720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc79c003fc0 0x7fc79c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7fc7b2ffd640 1 --2- 172.21.15.103:0/1831650720 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b41462b0 0x7fc79c003a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7f8279c63640 1 -- 172.21.15.103:0/856922849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f826c02afb0 msgr2=0x7f826c0c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7f8279c63640 1 --2- 172.21.15.103:0/856922849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f826c02afb0 0x7f826c0c6b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82740745e0 tx=0x7f82740a99e0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7fad7a513640 1 --2- 172.21.15.103:0/2487108182 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad5c0051c0 0x7fad5c00b9c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fad6c001e50 tx=0x7fad5c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f54ddb173fa2290d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7fcf32824640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fcf2c071540 0x7fcf2c1404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7fcf32824640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf2c072140 con 0x7fcf2c071160 2024-08-19T13:14:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7fcf32824640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf2c0ae530 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7fcf32824640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fcf2c073340 con 0x7fcf2c071540 2024-08-19T13:14:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7feff4d08640 1 -- start start 2024-08-19T13:14:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7f322922c640 1 --2- 172.21.15.103:0/4179237724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c0c0af0 0x7f321c0c0f40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3224069f60 tx=0x7f32240a5410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de769e188133b143 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7f8279c63640 1 -- 172.21.15.103:0/856922849 shutdown_connections 2024-08-19T13:14:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7f8279c63640 1 --2- 172.21.15.103:0/856922849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f826c02afb0 0x7f826c0c6b20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7f8279c63640 1 --2- 172.21.15.103:0/856922849 >> v2:172.21.15.122:3300/0 conn(0x7f826c0c0940 0x7f826c034490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7f8279c63640 1 -- 172.21.15.103:0/856922849 >> 172.21.15.103:0/856922849 conn(0x7f826c005a00 msgr2=0x7f826c005de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7f8279c63640 1 -- 172.21.15.103:0/856922849 shutdown_connections 2024-08-19T13:14:59.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7fcf32023640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fcf2c13d9d0 0x7fcf2c140a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.103:60046/0 (socket says 172.21.15.103:60046) 2024-08-19T13:14:59.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7fcf32023640 1 -- 172.21.15.103:0/2574335655 learned_addr learned my addr 172.21.15.103:0/2574335655 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:14:59.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7fcf31021640 1 --2- 172.21.15.103:0/2574335655 >> v2:172.21.15.122:3300/0 conn(0x7fcf2c071540 0x7fcf2c1404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7f32237fe640 1 -- 172.21.15.103:0/4179237724 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240b1070 con 0x7f321c0c0af0 2024-08-19T13:14:59.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7feff4d08640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feff00ac920 0x7feff0144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7feff4d08640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feff00ae450 0x7feff01470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7feff4d08640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff01406f0 0x7feff013f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7fefef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feff00ac920 0x7feff0144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7fefef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feff00ac920 0x7feff0144790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49862/0 (socket says 172.21.15.103:49862) 2024-08-19T13:14:59.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7fefef7fe640 1 -- 172.21.15.103:0/14083251 learned_addr learned my addr 172.21.15.103:0/14083251 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7fefeffff640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff01406f0 0x7feff013f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7feff4d08640 1 -- 172.21.15.103:0/14083251 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feff00a9e70 con 0x7feff00ac920 2024-08-19T13:14:59.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7fefeeffd640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feff00ae450 0x7feff01470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7feff4d08640 1 -- 172.21.15.103:0/14083251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feff00ae890 con 0x7feff01406f0 2024-08-19T13:14:59.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7feff4d08640 1 -- 172.21.15.103:0/14083251 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7feff00aea10 con 0x7feff00ae450 2024-08-19T13:14:59.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7fad78d10640 1 -- 172.21.15.103:0/2487108182 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64009690 con 0x7fad5c0051c0 2024-08-19T13:14:59.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7fad78d10640 1 -- 172.21.15.103:0/2487108182 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6c003970 con 0x7fad5c0051c0 2024-08-19T13:14:59.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7fad78d10640 1 -- 172.21.15.103:0/2487108182 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c001a70 con 0x7fad5c0051c0 2024-08-19T13:14:59.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7fc7b27fc640 1 -- 172.21.15.103:0/1831650720 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b41462b0 msgr2=0x7fc79c003a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7fc7b27fc640 1 --2- 172.21.15.103:0/1831650720 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b41462b0 0x7fc79c003a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7fc7b27fc640 1 -- 172.21.15.103:0/1831650720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc79c004d40 msgr2=0x7fc79c0051c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T13:14:59.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7fc7b27fc640 1 --2- 172.21.15.103:0/1831650720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc79c004d40 0x7fc79c0051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7fc7b27fc640 1 -- 172.21.15.103:0/1831650720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b414a1c0 con 0x7fc79c003fc0 2024-08-19T13:14:59.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7fc7b27fc640 1 --2- 172.21.15.103:0/1831650720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc79c003fc0 0x7fc79c004800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac007360 tx=0x7fc79c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae2a70921003d9b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7fec00cc7640 1 --2- 172.21.15.103:0/3476300621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febfc06f4e0 0x7febfc072c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7febf4009e80 tx=0x7febf402a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c7e609f27fb548d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7fcf31021640 1 -- 172.21.15.103:0/2574335655 >> v1:172.21.15.122:6789/0 conn(0x7fcf2c13d9d0 legacy=0x7fcf2c140a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7fcf31021640 1 -- 172.21.15.103:0/2574335655 >> v1:172.21.15.103:6789/0 conn(0x7fcf2c071160 legacy=0x7fcf2c13d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7fcf30820640 1 -- 172.21.15.103:0/2574335655 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2005726818 0 0) 0x7fcf2c0ae530 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7f32237fe640 1 -- 172.21.15.103:0/4179237724 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32240aa7d0 con 0x7f321c0c0af0 2024-08-19T13:14:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7f32237fe640 1 -- 172.21.15.103:0/4179237724 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240b1070 con 0x7f321c0c0af0 2024-08-19T13:14:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7fc7b1ffb640 1 -- 172.21.15.103:0/1831650720 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac009be0 con 0x7fc79c003fc0 2024-08-19T13:14:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7fc7b1ffb640 1 -- 172.21.15.103:0/1831650720 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7ac036600 con 0x7fc79c003fc0 2024-08-19T13:14:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7fc7b1ffb640 1 -- 172.21.15.103:0/1831650720 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac0369d0 con 0x7fc79c003fc0 2024-08-19T13:14:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7fad7bf76640 1 -- 172.21.15.103:0/2487108182 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad5c0051c0 msgr2=0x7fad5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7fad7bf76640 1 --2- 172.21.15.103:0/2487108182 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad5c0051c0 0x7fad5c00b9c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fad6c001e50 tx=0x7fad5c005830 comp rx=0 tx=0).stop 2024-08-19T13:14:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7f109d64e640 1 Processor -- start 2024-08-19T13:14:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7f3229a2d640 1 -- 172.21.15.103:0/4179237724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c0c0af0 msgr2=0x7f321c0c0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7f3229a2d640 1 --2- 172.21.15.103:0/4179237724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c0c0af0 0x7f321c0c0f40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3224069f60 tx=0x7f32240a5410 comp rx=0 tx=0).stop 2024-08-19T13:14:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7f109d64e640 1 -- start start 2024-08-19T13:14:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.558+0000 7f8279c63640 1 -- 172.21.15.103:0/856922849 wait complete. 2024-08-19T13:14:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7f8279c63640 1 Processor -- start 2024-08-19T13:14:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7f8279c63640 1 -- start start 2024-08-19T13:14:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7fefeffff640 1 -- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feff00ae450 msgr2=0x7feff01470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7f3229a2d640 1 -- 172.21.15.103:0/4179237724 shutdown_connections 2024-08-19T13:14:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7f3229a2d640 1 --2- 172.21.15.103:0/4179237724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c0c0af0 0x7f321c0c0f40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7f3229a2d640 1 -- 172.21.15.103:0/4179237724 >> 172.21.15.103:0/4179237724 conn(0x7f321c0066a0 msgr2=0x7f321c006ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7febf3fff640 1 -- 172.21.15.103:0/3476300621 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4036070 con 0x7febfc06f4e0 2024-08-19T13:14:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7febf3fff640 1 -- 172.21.15.103:0/3476300621 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf4031600 con 0x7febfc06f4e0 2024-08-19T13:14:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.559+0000 7febf3fff640 1 -- 172.21.15.103:0/3476300621 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf40319d0 con 0x7febfc06f4e0 2024-08-19T13:14:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7fefeffff640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feff00ae450 0x7feff01470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fc7b3fff640 1 -- 172.21.15.103:0/1831650720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc79c003fc0 msgr2=0x7fc79c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fc7b3fff640 1 --2- 172.21.15.103:0/1831650720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc79c003fc0 0x7fc79c004800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac007360 tx=0x7fc79c005830 comp rx=0 tx=0).stop 2024-08-19T13:14:59.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f3229a2d640 1 -- 172.21.15.103:0/4179237724 shutdown_connections 2024-08-19T13:14:59.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fad7bf76640 1 -- 172.21.15.103:0/2487108182 shutdown_connections 2024-08-19T13:14:59.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fad7bf76640 1 --2- 172.21.15.103:0/2487108182 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad5c0051c0 0x7fad5c00b9c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f3229a2d640 1 -- 172.21.15.103:0/4179237724 wait complete. 2024-08-19T13:14:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fad7bf76640 1 -- 172.21.15.103:0/2487108182 >> 172.21.15.103:0/2487108182 conn(0x7fad740b41c0 msgr2=0x7fad740b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7fefeffff640 1 -- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feff00ac920 msgr2=0x7feff0144790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7fefeffff640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feff00ac920 0x7feff0144790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7fefeffff640 1 -- 172.21.15.103:0/14083251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff0148010 con 0x7feff01406f0 2024-08-19T13:14:59.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7fefef7fe640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feff00ac920 0x7feff0144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:59.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fefeeffd640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feff00ae450 0x7feff01470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:59.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.560+0000 7fcf31021640 1 -- 172.21.15.103:0/2574335655 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c073620 con 0x7fcf2c071540 2024-08-19T13:14:59.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fec0372c640 1 -- 172.21.15.103:0/3476300621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febfc06f4e0 msgr2=0x7febfc072c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fec0372c640 1 --2- 172.21.15.103:0/3476300621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febfc06f4e0 0x7febfc072c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7febf4009e80 tx=0x7febf402a6d0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fec0372c640 1 -- 172.21.15.103:0/3476300621 shutdown_connections 2024-08-19T13:14:59.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fec0372c640 1 --2- 172.21.15.103:0/3476300621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febfc06f4e0 0x7febfc072c50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fec0372c640 1 -- 172.21.15.103:0/3476300621 >> 172.21.15.103:0/3476300621 conn(0x7febfc0b80d0 msgr2=0x7febfc0b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fad7bf76640 1 -- 172.21.15.103:0/2487108182 shutdown_connections 2024-08-19T13:14:59.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fc7b37fe640 1 -- 172.21.15.103:0/1831650720 reap_dead start 2024-08-19T13:14:59.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fc7b3fff640 1 -- 172.21.15.103:0/1831650720 shutdown_connections 2024-08-19T13:14:59.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fc7b3fff640 1 -- 172.21.15.103:0/1831650720 >> 172.21.15.103:0/1831650720 conn(0x7fc7b40b3eb0 msgr2=0x7fc7b40b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fad7bf76640 1 -- 172.21.15.103:0/2487108182 wait complete. 2024-08-19T13:14:59.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f109d64e640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f10980aec80 0x7f109813c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f109d64e640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1098072f60 0x7f10980703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f109d64e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10980712a0 con 0x7f109813cb20 2024-08-19T13:14:59.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f109d64e640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f10980af060 con 0x7f10980aec80 2024-08-19T13:14:59.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f109d64e640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f1098073a10 con 0x7f1098072f60 2024-08-19T13:14:59.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f1096ffd640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f10980aec80 0x7f109813c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f10977fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f109813cb20 0x7f10980708e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.103:60048/0 (socket says 172.21.15.103:60048) 2024-08-19T13:14:59.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fefeffff640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff01406f0 0x7feff013f600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fefe800cad0 tx=0x7fefe800cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f10977fe640 1 -- 172.21.15.103:0/2754173589 learned_addr learned my addr 172.21.15.103:0/2754173589 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:14:59.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7fefd7fff640 1 -- 172.21.15.103:0/14083251 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8017070 con 0x7feff01406f0 2024-08-19T13:14:59.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7f10967fc640 1 --2- 172.21.15.103:0/2754173589 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1098072f60 0x7f10980703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7fefd7fff640 1 -- 172.21.15.103:0/14083251 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe800ab70 con 0x7feff01406f0 2024-08-19T13:14:59.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7feff4d08640 1 -- 172.21.15.103:0/14083251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff01482f0 con 0x7feff01406f0 2024-08-19T13:14:59.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f1096ffd640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f10980aec80 0x7f109813c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49876/0 (socket says 172.21.15.103:49876) 2024-08-19T13:14:59.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7feff4d08640 1 -- 172.21.15.103:0/14083251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff01419c0 con 0x7feff01406f0 2024-08-19T13:14:59.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7fefd7fff640 1 -- 172.21.15.103:0/14083251 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8012400 con 0x7feff01406f0 2024-08-19T13:14:59.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7fc7b3fff640 1 -- 172.21.15.103:0/1831650720 shutdown_connections 2024-08-19T13:14:59.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7fc7b3fff640 1 -- 172.21.15.103:0/1831650720 wait complete. 2024-08-19T13:14:59.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f8279c63640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f826c02afb0 0x7f826c0b8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f8279c63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f826c02b3d0 0x7f826c02cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f8279c63640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f826c0c0940 0x7f826c02d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f8279c63640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f826c0349d0 con 0x7f826c0c0940 2024-08-19T13:14:59.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f8279c63640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f826c0c10e0 con 0x7f826c02b3d0 2024-08-19T13:14:59.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f8279c63640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f826c02a820 con 0x7f826c02afb0 2024-08-19T13:14:59.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f8273fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f826c02b3d0 0x7f826c02cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f8273fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f826c02b3d0 0x7f826c02cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.103:48758/0 (socket says 172.21.15.103:48758) 2024-08-19T13:14:59.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7f8273fff640 1 -- 172.21.15.103:0/1844425570 learned_addr learned my addr 172.21.15.103:0/1844425570 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7f10967fc640 1 -- 172.21.15.103:0/2754173589 >> v2:172.21.15.103:3300/0 conn(0x7f10980aec80 msgr2=0x7f109813c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:14:59.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fec0372c640 1 -- 172.21.15.103:0/3476300621 shutdown_connections 2024-08-19T13:14:59.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.561+0000 7fec0372c640 1 -- 172.21.15.103:0/3476300621 wait complete. 2024-08-19T13:14:59.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7fcf31021640 1 --2- 172.21.15.103:0/2574335655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf2c071540 0x7fcf2c1404f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcf280097a0 tx=0x7fcf2802a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ee4670baec309f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7f8278c61640 1 --2- 172.21.15.103:0/1844425570 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f826c02afb0 0x7f826c0b8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7f8273fff640 1 -- 172.21.15.103:0/1844425570 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f826c02afb0 msgr2=0x7f826c0b8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7f8273fff640 1 --2- 172.21.15.103:0/1844425570 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f826c02afb0 0x7f826c0b8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7f8273fff640 1 -- 172.21.15.103:0/1844425570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f826c0c0940 msgr2=0x7f826c02d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7f8273fff640 1 --2- 172.21.15.103:0/1844425570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f826c0c0940 0x7f826c02d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7f8273fff640 1 -- 172.21.15.103:0/1844425570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c02eb80 con 0x7f826c02b3d0 2024-08-19T13:14:59.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.562+0000 7f8278c61640 1 --2- 172.21.15.103:0/1844425570 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f826c02afb0 0x7f826c0b8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T13:14:59.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7f1095ffb640 1 -- 172.21.15.103:0/2754173589 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1710136243 0 0) 0x7f10980712a0 con 0x7f109813cb20 2024-08-19T13:14:59.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7f10967fc640 1 --2- 172.21.15.103:0/2754173589 >> v2:172.21.15.103:3300/0 conn(0x7f10980aec80 0x7f109813c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7f10967fc640 1 -- 172.21.15.103:0/2754173589 >> v1:172.21.15.122:6789/0 conn(0x7f109813cb20 legacy=0x7f10980708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7f10967fc640 1 -- 172.21.15.103:0/2754173589 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10980716f0 con 0x7f1098072f60 2024-08-19T13:14:59.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7fad7bf76640 1 Processor -- start 2024-08-19T13:14:59.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7f1096ffd640 1 --2- 172.21.15.103:0/2754173589 >> v2:172.21.15.103:3300/0 conn(0x7f10980aec80 0x7f109813c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:59.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7f3229a2d640 1 Processor -- start 2024-08-19T13:14:59.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7fad7bf76640 1 -- start start 2024-08-19T13:14:59.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7fc7b3fff640 1 Processor -- start 2024-08-19T13:14:59.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7f3229a2d640 1 -- start start 2024-08-19T13:14:59.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7fec0372c640 1 Processor -- start 2024-08-19T13:14:59.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7fec0372c640 1 -- start start 2024-08-19T13:14:59.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7fcf30820640 1 -- 172.21.15.103:0/2574335655 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28037070 con 0x7fcf2c071540 2024-08-19T13:14:59.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7fcf30820640 1 -- 172.21.15.103:0/2574335655 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2802ad30 con 0x7fcf2c071540 2024-08-19T13:14:59.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7fcf30820640 1 -- 172.21.15.103:0/2574335655 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28032660 con 0x7fcf2c071540 2024-08-19T13:14:59.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.563+0000 7fc7b3fff640 1 -- start start 2024-08-19T13:14:59.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fefd7fff640 1 -- 172.21.15.103:0/14083251 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fefe801a070 con 0x7feff01406f0 2024-08-19T13:14:59.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fec0372c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febfc072410 0x7febfc14d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fec0372c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febfc14f450 0x7febfc151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fec0372c640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7febfc14ef20 0x7febfc151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fec0372c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7febfc150170 con 0x7febfc14f450 2024-08-19T13:14:59.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fec0372c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7febfc14fff0 con 0x7febfc072410 2024-08-19T13:14:59.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fec0372c640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7febfc15ab40 con 0x7febfc14ef20 2024-08-19T13:14:59.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7f8273fff640 1 --2- 172.21.15.103:0/1844425570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f826c02b3d0 0x7f826c02cc10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f827406b450 tx=0x7f827406a7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7f10967fc640 1 --2- 172.21.15.103:0/2754173589 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1098072f60 0x7f10980703a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f108c00ba40 tx=0x7f108c02c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b826d7bb1ba77b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fefd7fff640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fefdc06bf00 0x7fefdc06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fefd7fff640 1 -- 172.21.15.103:0/14083251 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7fefe808a890 con 0x7feff01406f0 2024-08-19T13:14:59.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fad7bf76640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c0051c0 0x7fad7413eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fad7bf76640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad7412b650 0x7fad740a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b40aa4e0 0x7fc7b40aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7b413e790 0x7fc7b413ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b41db9b0 0x7fc7b41dbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fad79d12640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c0051c0 0x7fad7413eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fad79d12640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c0051c0 0x7fad7413eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.103:48776/0 (socket says 172.21.15.103:48776) 2024-08-19T13:14:59.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fc7b2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b40aa4e0 0x7fc7b40aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fc7b2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b40aa4e0 0x7fc7b40aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49914/0 (socket says 172.21.15.103:49914) 2024-08-19T13:14:59.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fefef7fe640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fefdc06bf00 0x7fefdc06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fad79d12640 1 -- 172.21.15.103:0/1743776220 learned_addr learned my addr 172.21.15.103:0/1743776220 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fc7b3fff640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4073420 con 0x7fc7b40aa4e0 2024-08-19T13:14:59.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fad7bf76640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad741412a0 0x7fad74141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fc7b3fff640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b40732a0 con 0x7fc7b413e790 2024-08-19T13:14:59.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fc7b2ffd640 1 -- 172.21.15.103:0/2362843535 learned_addr learned my addr 172.21.15.103:0/2362843535 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fc7b3fff640 1 -- 172.21.15.103:0/2362843535 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b414a390 con 0x7fc7b41db9b0 2024-08-19T13:14:59.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7f1095ffb640 1 -- 172.21.15.103:0/2754173589 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c032750 con 0x7f1098072f60 2024-08-19T13:14:59.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7f82717fa640 1 -- 172.21.15.103:0/1844425570 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82740bb070 con 0x7f826c02b3d0 2024-08-19T13:14:59.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7f82717fa640 1 -- 172.21.15.103:0/1844425570 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82740addd0 con 0x7f826c02b3d0 2024-08-19T13:14:59.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7f82717fa640 1 -- 172.21.15.103:0/1844425570 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82740b2040 con 0x7f826c02b3d0 2024-08-19T13:14:59.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7feff4d08640 1 -- 172.21.15.103:0/14083251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff0141b90 con 0x7feff01406f0 2024-08-19T13:14:59.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7f8279c63640 1 -- 172.21.15.103:0/1844425570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826c02ee60 con 0x7f826c02b3d0 2024-08-19T13:14:59.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7f3229a2d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f321c034f90 0x7f321c030280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7f8279c63640 1 -- 172.21.15.103:0/1844425570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826c031bc0 con 0x7f826c02b3d0 2024-08-19T13:14:59.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fad7a513640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad741412a0 0x7fad74141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fad7bf76640 1 -- 172.21.15.103:0/1743776220 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad7413b4b0 con 0x7fad741412a0 2024-08-19T13:14:59.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fad7bf76640 1 -- 172.21.15.103:0/1743776220 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fad74071ea0 con 0x7fad5c0051c0 2024-08-19T13:14:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fad7bf76640 1 -- 172.21.15.103:0/1743776220 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fad7413b630 con 0x7fad7412b650 2024-08-19T13:14:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fad79511640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad7412b650 0x7fad740a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fc7b27fc640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7b413e790 0x7fc7b413ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fcf32824640 1 -- 172.21.15.103:0/2574335655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf2c071540 msgr2=0x7fcf2c1404f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fcf32824640 1 --2- 172.21.15.103:0/2574335655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf2c071540 0x7fcf2c1404f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcf280097a0 tx=0x7fcf2802a0a0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fcf32824640 1 -- 172.21.15.103:0/2574335655 shutdown_connections 2024-08-19T13:14:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fcf32824640 1 --2- 172.21.15.103:0/2574335655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf2c071540 0x7fcf2c1404f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fcf32824640 1 -- 172.21.15.103:0/2574335655 >> 172.21.15.103:0/2574335655 conn(0x7fcf2c0b0c60 msgr2=0x7fcf2c06b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fad79511640 1 -- 172.21.15.103:0/1743776220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c0051c0 msgr2=0x7fad7413eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7fefef7fe640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fefdc06bf00 0x7fefdc06e3c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fefe00097a0 tx=0x7fefe002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fad79511640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c0051c0 0x7fad7413eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3229a2d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f321c0c0af0 0x7f321c02f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3229a2d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c0c4610 0x7f321c0bc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3229a2d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f321c0c7730 con 0x7f321c034f90 2024-08-19T13:14:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7fcf32824640 1 -- 172.21.15.103:0/2574335655 shutdown_connections 2024-08-19T13:14:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3229a2d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f321c0c75b0 con 0x7f321c0c4610 2024-08-19T13:14:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3229a2d640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f321c0cf240 con 0x7f321c0c0af0 2024-08-19T13:14:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7fcf32824640 1 -- 172.21.15.103:0/2574335655 wait complete. 2024-08-19T13:14:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fc7b37fe640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b41db9b0 0x7fc7b41dbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fc7b2ffd640 1 -- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b41db9b0 msgr2=0x7fc7b41dbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fc7b2ffd640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b41db9b0 0x7fc7b41dbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fc7b2ffd640 1 -- 172.21.15.103:0/2362843535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7b413e790 msgr2=0x7fc7b413ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fc7b2ffd640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7b413e790 0x7fc7b413ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fc7b2ffd640 1 -- 172.21.15.103:0/2362843535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b40a9d90 con 0x7fc7b40aa4e0 2024-08-19T13:14:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7fc7b37fe640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b41db9b0 0x7fc7b41dbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T13:14:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7fc7b27fc640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7b413e790 0x7fc7b413ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T13:14:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7fc7b2ffd640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b40aa4e0 0x7fc7b40aa8c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400eb40 tx=0x7fc7a4005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f1095ffb640 1 -- 172.21.15.103:0/2754173589 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f108c030050 con 0x7f1098072f60 2024-08-19T13:14:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f1095ffb640 1 -- 172.21.15.103:0/2754173589 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c032b10 con 0x7f1098072f60 2024-08-19T13:14:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3223fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f321c0c0af0 0x7f321c02f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3228a2b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f321c034f90 0x7f321c030280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3228a2b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f321c034f90 0x7f321c030280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49936/0 (socket says 172.21.15.103:49936) 2024-08-19T13:14:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3223fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f321c0c0af0 0x7f321c02f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48748/0 (socket says 172.21.15.103:48748) 2024-08-19T13:14:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3223fff640 1 -- 172.21.15.103:0/1944317711 learned_addr learned my addr 172.21.15.103:0/1944317711 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f8279c63640 1 -- 172.21.15.103:0/1844425570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826c0c0d20 con 0x7f826c02b3d0 2024-08-19T13:14:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7fc79b7fe640 1 -- 172.21.15.103:0/2362843535 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a400c8d0 con 0x7fc7b40aa4e0 2024-08-19T13:14:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7fc79b7fe640 1 -- 172.21.15.103:0/2362843535 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a4018070 con 0x7fc7b40aa4e0 2024-08-19T13:14:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7fc7b3fff640 1 -- 172.21.15.103:0/2362843535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b40abac0 con 0x7fc7b40aa4e0 2024-08-19T13:14:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7fc79b7fe640 1 -- 172.21.15.103:0/2362843535 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a400cc70 con 0x7fc7b40aa4e0 2024-08-19T13:14:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7fc7b3fff640 1 -- 172.21.15.103:0/2362843535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b413f130 con 0x7fc7b40aa4e0 2024-08-19T13:14:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fec01cc9640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7febfc14ef20 0x7febfc151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fec01cc9640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7febfc14ef20 0x7febfc151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48706/0 (socket says 172.21.15.103:48706) 2024-08-19T13:14:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fec01cc9640 1 -- 172.21.15.103:0/3720851542 learned_addr learned my addr 172.21.15.103:0/3720851542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fec01cc9640 1 -- 172.21.15.103:0/3720851542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febfc072410 msgr2=0x7febfc14d1b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T13:14:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fec01cc9640 1 --2- 172.21.15.103:0/3720851542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febfc072410 0x7febfc14d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fec01cc9640 1 -- 172.21.15.103:0/3720851542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febfc14f450 msgr2=0x7febfc151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fec01cc9640 1 --2- 172.21.15.103:0/3720851542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febfc14f450 0x7febfc151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.564+0000 7fec01cc9640 1 -- 172.21.15.103:0/3720851542 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febfc145420 con 0x7febfc14ef20 2024-08-19T13:14:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f322922c640 1 --2- 172.21.15.103:0/1944317711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c0c4610 0x7f321c0bc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3223fff640 1 -- 172.21.15.103:0/1944317711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c0c4610 msgr2=0x7f321c0bc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3223fff640 1 --2- 172.21.15.103:0/1944317711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c0c4610 0x7f321c0bc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3223fff640 1 -- 172.21.15.103:0/1944317711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f321c034f90 msgr2=0x7f321c030280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3223fff640 1 --2- 172.21.15.103:0/1944317711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f321c034f90 0x7f321c030280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7f3223fff640 1 -- 172.21.15.103:0/1944317711 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c031250 con 0x7f321c0c0af0 2024-08-19T13:14:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.567+0000 7f3223fff640 1 --2- 172.21.15.103:0/1944317711 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f321c0c0af0 0x7f321c02f700 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f321800c3c0 tx=0x7f321800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.567+0000 7fec01cc9640 1 --2- 172.21.15.103:0/3720851542 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7febfc14ef20 0x7febfc151870 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7febec00c550 tx=0x7febec00ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.565+0000 7fad79511640 1 -- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad741412a0 msgr2=0x7fad74141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7fad79511640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad741412a0 0x7fad74141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.566+0000 7fad79511640 1 -- 172.21.15.103:0/1743776220 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad740a9780 con 0x7fad7412b650 2024-08-19T13:14:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.567+0000 7fad79511640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad7412b650 0x7fad740a9ba0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fad6800c760 tx=0x7fad6800cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.567+0000 7fcf32824640 1 Processor -- start 2024-08-19T13:14:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.567+0000 7f109d64e640 1 -- 172.21.15.103:0/2754173589 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1098072f60 msgr2=0x7f10980703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.567+0000 7f109d64e640 1 --2- 172.21.15.103:0/2754173589 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1098072f60 0x7f10980703a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f108c00ba40 tx=0x7f108c02c850 comp rx=0 tx=0).stop 2024-08-19T13:14:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.567+0000 7fcf32824640 1 -- start start 2024-08-19T13:14:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.567+0000 7fad7a513640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad741412a0 0x7fad74141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:14:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.567+0000 7fad79d12640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c0051c0 0x7fad7413eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T13:14:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fad727fc640 1 -- 172.21.15.103:0/1743776220 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68018070 con 0x7fad7412b650 2024-08-19T13:14:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fad727fc640 1 -- 172.21.15.103:0/1743776220 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad680072d0 con 0x7fad7412b650 2024-08-19T13:14:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fad727fc640 1 -- 172.21.15.103:0/1743776220 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad680047e0 con 0x7fad7412b650 2024-08-19T13:14:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7f109d64e640 1 -- 172.21.15.103:0/2754173589 shutdown_connections 2024-08-19T13:14:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7f109d64e640 1 --2- 172.21.15.103:0/2754173589 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1098072f60 0x7f10980703a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7f109d64e640 1 --2- 172.21.15.103:0/2754173589 >> v2:172.21.15.103:3300/0 conn(0x7f10980aec80 0x7f109813c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7f109d64e640 1 -- 172.21.15.103:0/2754173589 >> 172.21.15.103:0/2754173589 conn(0x7f10980b41c0 msgr2=0x7f10980b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7f109d64e640 1 -- 172.21.15.103:0/2754173589 shutdown_connections 2024-08-19T13:14:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7f109d64e640 1 -- 172.21.15.103:0/2754173589 wait complete. 2024-08-19T13:14:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7febf1ffb640 1 -- 172.21.15.103:0/3720851542 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec016020 con 0x7febfc14ef20 2024-08-19T13:14:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7febf1ffb640 1 -- 172.21.15.103:0/3720851542 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febec0052d0 con 0x7febfc14ef20 2024-08-19T13:14:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7febf1ffb640 1 -- 172.21.15.103:0/3720851542 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec016020 con 0x7febfc14ef20 2024-08-19T13:14:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf32824640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf2c071160 0x7fcf2c135110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf32824640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf2c071540 0x7fcf2c135950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf32824640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf2c13d9d0 0x7fcf2c0b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf32824640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c06eb20 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf32824640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c06e9a0 con 0x7fcf2c071160 2024-08-19T13:14:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf32824640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2c06eca0 con 0x7fcf2c071540 2024-08-19T13:14:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf32023640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf2c13d9d0 0x7fcf2c0b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf32023640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf2c13d9d0 0x7fcf2c0b1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49952/0 (socket says 172.21.15.103:49952) 2024-08-19T13:14:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf32023640 1 -- 172.21.15.103:0/3795937062 learned_addr learned my addr 172.21.15.103:0/3795937062 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7f32217fa640 1 -- 172.21.15.103:0/1944317711 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32180052d0 con 0x7f321c0c0af0 2024-08-19T13:14:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7f32217fa640 1 -- 172.21.15.103:0/1944317711 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3218005470 con 0x7f321c0c0af0 2024-08-19T13:14:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7f3229a2d640 1 -- 172.21.15.103:0/1944317711 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321c031530 con 0x7f321c0c0af0 2024-08-19T13:14:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf31822640 1 --2- 172.21.15.103:0/3795937062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf2c071160 0x7fcf2c135110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7f32217fa640 1 -- 172.21.15.103:0/1944317711 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218011840 con 0x7f321c0c0af0 2024-08-19T13:14:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fad7bf76640 1 -- 172.21.15.103:0/1743776220 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad7413f060 con 0x7fad7412b650 2024-08-19T13:14:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7f3229a2d640 1 -- 172.21.15.103:0/1944317711 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f321c0bca10 con 0x7f321c0c0af0 2024-08-19T13:14:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fad7bf76640 1 -- 172.21.15.103:0/1743776220 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7413da10 con 0x7fad7412b650 2024-08-19T13:14:59.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf32023640 1 -- 172.21.15.103:0/3795937062 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf2c071540 msgr2=0x7fcf2c135950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf32023640 1 --2- 172.21.15.103:0/3795937062 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf2c071540 0x7fcf2c135950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf32023640 1 -- 172.21.15.103:0/3795937062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf2c071160 msgr2=0x7fcf2c135110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf32023640 1 --2- 172.21.15.103:0/3795937062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf2c071160 0x7fcf2c135110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.568+0000 7fcf32023640 1 -- 172.21.15.103:0/3795937062 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c134ca0 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.569+0000 7fcf32023640 1 --2- 172.21.15.103:0/3795937062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf2c13d9d0 0x7fcf2c0b1bd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c001eb0 tx=0x7fcf1c00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.569+0000 7fcf227fc640 1 -- 172.21.15.103:0/3795937062 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0074e0 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.569+0000 7fec0372c640 1 -- 172.21.15.103:0/3720851542 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febfc151db0 con 0x7febfc14ef20 2024-08-19T13:14:59.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.569+0000 7fcf32824640 1 -- 172.21.15.103:0/3795937062 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2c1364f0 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.569+0000 7fec0372c640 1 -- 172.21.15.103:0/3720851542 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7febfc14eb00 con 0x7febfc14ef20 2024-08-19T13:14:59.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.569+0000 7fcf32824640 1 -- 172.21.15.103:0/3795937062 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2c0b2110 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.570+0000 7f109d64e640 1 Processor -- start 2024-08-19T13:14:59.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.570+0000 7fad7bf76640 1 -- 172.21.15.103:0/1743776220 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad740703b0 con 0x7fad7412b650 2024-08-19T13:14:59.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.570+0000 7f109d64e640 1 -- start start 2024-08-19T13:14:59.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.570+0000 7f3229a2d640 1 -- 172.21.15.103:0/1944317711 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321c0c0f40 con 0x7f321c0c0af0 2024-08-19T13:14:59.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.569+0000 7fc79b7fe640 1 -- 172.21.15.103:0/2362843535 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc7a401d460 con 0x7fc7b40aa4e0 2024-08-19T13:14:59.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.570+0000 7fc7b3fff640 1 -- 172.21.15.103:0/2362843535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b4149c40 con 0x7fc7b40aa4e0 2024-08-19T13:14:59.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.570+0000 7fc79b7fe640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc7a006bf30 0x7fc7a006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.570+0000 7fc79b7fe640 1 -- 172.21.15.103:0/2362843535 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7fc7a408ac20 con 0x7fc7b40aa4e0 2024-08-19T13:14:59.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.570+0000 7fec0372c640 1 -- 172.21.15.103:0/3720851542 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febfc0713e0 con 0x7febfc14ef20 2024-08-19T13:14:59.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.570+0000 7fcf227fc640 1 -- 172.21.15.103:0/3795937062 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf1c0047e0 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.571+0000 7fcf227fc640 1 -- 172.21.15.103:0/3795937062 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c004bb0 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.571+0000 7fefd7fff640 1 -- 172.21.15.103:0/14083251 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fefe8058e10 con 0x7feff01406f0 2024-08-19T13:14:59.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.571+0000 7fad727fc640 1 -- 172.21.15.103:0/1743776220 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad6801b070 con 0x7fad7412b650 2024-08-19T13:14:59.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.571+0000 7fcf0ffff640 1 -- 172.21.15.103:0/3795937062 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf08005080 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.571+0000 7f109d64e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1098072f60 0x7f1098140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.571+0000 7fc7b27fc640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc7a006bf30 0x7fc7a006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.572+0000 7fad727fc640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fad6006bf00 0x7fad6006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.572+0000 7fad79d12640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fad6006bf00 0x7fad6006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.572+0000 7fad727fc640 1 -- 172.21.15.103:0/1743776220 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7fad68089ff0 con 0x7fad7412b650 2024-08-19T13:14:59.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.572+0000 7f109d64e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10980aec80 0x7f1098143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.572+0000 7f109d64e640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f109813cb20 0x7f10981e3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.572+0000 7fc7b27fc640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc7a006bf30 0x7fc7a006e3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac035000 tx=0x7fc7ac00b030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.572+0000 7f32217fa640 1 -- 172.21.15.103:0/1944317711 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f32180119e0 con 0x7f321c0c0af0 2024-08-19T13:14:59.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.572+0000 7fad79d12640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fad6006bf00 0x7fad6006e3c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fad64009c40 tx=0x7fad64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.572+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1098072f60 0x7f1098140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.572+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1098072f60 0x7f1098140ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49962/0 (socket says 172.21.15.103:49962) 2024-08-19T13:14:59.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.573+0000 7f1096ffd640 1 -- 172.21.15.103:0/3786692694 learned_addr learned my addr 172.21.15.103:0/3786692694 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:14:59.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.573+0000 7f109d64e640 1 -- 172.21.15.103:0/3786692694 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10980af2a0 con 0x7f1098072f60 2024-08-19T13:14:59.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.573+0000 7f109d64e640 1 -- 172.21.15.103:0/3786692694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1098073550 con 0x7f10980aec80 2024-08-19T13:14:59.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.573+0000 7f109d64e640 1 -- 172.21.15.103:0/3786692694 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f10980af420 con 0x7f109813cb20 2024-08-19T13:14:59.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.573+0000 7f32217fa640 1 --2- 172.21.15.103:0/1944317711 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f321006bf00 0x7f321006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.573+0000 7f10967fc640 1 --2- 172.21.15.103:0/3786692694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10980aec80 0x7f1098143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.573+0000 7f10977fe640 1 --2- 172.21.15.103:0/3786692694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f109813cb20 0x7f10981e3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.573+0000 7f32217fa640 1 -- 172.21.15.103:0/1944317711 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f321808c310 con 0x7f321c0c0af0 2024-08-19T13:14:59.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.573+0000 7f3228a2b640 1 --2- 172.21.15.103:0/1944317711 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f321006bf00 0x7f321006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.573+0000 7f82717fa640 1 -- 172.21.15.103:0/1844425570 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82740be070 con 0x7f826c02b3d0 2024-08-19T13:14:59.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.574+0000 7febf1ffb640 1 -- 172.21.15.103:0/3720851542 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7febec007020 con 0x7febfc14ef20 2024-08-19T13:14:59.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.574+0000 7f1096ffd640 1 -- 172.21.15.103:0/3786692694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f109813cb20 msgr2=0x7f10981e3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.574+0000 7f1096ffd640 1 --2- 172.21.15.103:0/3786692694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f109813cb20 0x7f10981e3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.574+0000 7f1096ffd640 1 -- 172.21.15.103:0/3786692694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10980aec80 msgr2=0x7f1098143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.574+0000 7f1096ffd640 1 --2- 172.21.15.103:0/3786692694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10980aec80 0x7f1098143090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.574+0000 7f1096ffd640 1 -- 172.21.15.103:0/3786692694 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1098142900 con 0x7f1098072f60 2024-08-19T13:14:59.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.574+0000 7f82717fa640 1 --2- 172.21.15.103:0/1844425570 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f825806bf00 0x7f825806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.574+0000 7f82717fa640 1 -- 172.21.15.103:0/1844425570 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f8274126cb0 con 0x7f826c02b3d0 2024-08-19T13:14:59.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.574+0000 7f1096ffd640 1 --2- 172.21.15.103:0/3786692694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1098072f60 0x7f1098140ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f108800ebe0 tx=0x7f108800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.574+0000 7f3228a2b640 1 --2- 172.21.15.103:0/1944317711 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f321006bf00 0x7f321006e3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3214000f60 tx=0x7f3214007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.574+0000 7febf1ffb640 1 --2- 172.21.15.103:0/3720851542 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7febe006c230 0x7febe006e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.574+0000 7f8278c61640 1 --2- 172.21.15.103:0/1844425570 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f825806bf00 0x7f825806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.574+0000 7febf1ffb640 1 -- 172.21.15.103:0/3720851542 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7febec0050d0 con 0x7febfc14ef20 2024-08-19T13:14:59.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.574+0000 7fec014c8640 1 --2- 172.21.15.103:0/3720851542 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7febe006c230 0x7febe006e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.575+0000 7f8278c61640 1 --2- 172.21.15.103:0/1844425570 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f825806bf00 0x7f825806e3c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8268006fd0 tx=0x7f8268008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.575+0000 7fec014c8640 1 --2- 172.21.15.103:0/3720851542 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7febe006c230 0x7febe006e6f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7febf80014e0 tx=0x7febf80079d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.576+0000 7f82717fa640 1 -- 172.21.15.103:0/1844425570 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f8274090070 con 0x7f826c02b3d0 2024-08-19T13:14:59.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.577+0000 7fad727fc640 1 -- 172.21.15.103:0/1743776220 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fad68058570 con 0x7fad7412b650 2024-08-19T13:14:59.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.577+0000 7fc79b7fe640 1 -- 172.21.15.103:0/2362843535 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fc7a40597f0 con 0x7fc7b40aa4e0 2024-08-19T13:14:59.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.578+0000 7f107f7fe640 1 -- 172.21.15.103:0/3786692694 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088017070 con 0x7f1098072f60 2024-08-19T13:14:59.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.578+0000 7f109d64e640 1 -- 172.21.15.103:0/3786692694 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10981405c0 con 0x7f1098072f60 2024-08-19T13:14:59.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.578+0000 7f109d64e640 1 -- 172.21.15.103:0/3786692694 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1098140040 con 0x7f1098072f60 2024-08-19T13:14:59.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.580+0000 7fcf227fc640 1 -- 172.21.15.103:0/3795937062 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf1c011070 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.580+0000 7f107f7fe640 1 -- 172.21.15.103:0/3786692694 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f108800cc30 con 0x7f1098072f60 2024-08-19T13:14:59.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.580+0000 7f107f7fe640 1 -- 172.21.15.103:0/3786692694 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088021060 con 0x7f1098072f60 2024-08-19T13:14:59.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.580+0000 7fcf227fc640 1 --2- 172.21.15.103:0/3795937062 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fcf1006bf00 0x7fcf1006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.581+0000 7f107d7fa640 1 -- 172.21.15.103:0/3786692694 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1098069460 con 0x7f1098072f60 2024-08-19T13:14:59.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.581+0000 7fcf31822640 1 --2- 172.21.15.103:0/3795937062 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fcf1006bf00 0x7fcf1006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.582+0000 7fcf227fc640 1 -- 172.21.15.103:0/3795937062 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7fcf1c08aea0 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.582+0000 7fcf31822640 1 --2- 172.21.15.103:0/3795937062 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fcf1006bf00 0x7fcf1006e3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcf24007fd0 tx=0x7fcf24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.585+0000 7fcf227fc640 1 -- 172.21.15.103:0/3795937062 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fcf2c134ca0 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.587+0000 7febf1ffb640 1 -- 172.21.15.103:0/3720851542 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7febfc0713e0 con 0x7febfc14ef20 2024-08-19T13:14:59.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.586+0000 7f107f7fe640 1 -- 172.21.15.103:0/3786692694 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f108801a070 con 0x7f1098072f60 2024-08-19T13:14:59.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.587+0000 7f107f7fe640 1 --2- 172.21.15.103:0/3786692694 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f108406c260 0x7f108406e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:14:59.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.587+0000 7f107f7fe640 1 -- 172.21.15.103:0/3786692694 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f108800f040 con 0x7f1098072f60 2024-08-19T13:14:59.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.588+0000 7f10967fc640 1 --2- 172.21.15.103:0/3786692694 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f108406c260 0x7f108406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:14:59.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.588+0000 7f10967fc640 1 --2- 172.21.15.103:0/3786692694 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f108406c260 0x7f108406e720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f108c037000 tx=0x7f108c032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:14:59.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.593+0000 7f107f7fe640 1 -- 172.21.15.103:0/3786692694 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f1088059320 con 0x7f1098072f60 2024-08-19T13:14:59.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.595+0000 7f32217fa640 1 -- 172.21.15.103:0/1944317711 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f321805a890 con 0x7f321c0c0af0 2024-08-19T13:14:59.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.972+0000 7feff4d08640 1 -- 172.21.15.103:0/14083251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7feff006ed20 con 0x7feff01406f0 2024-08-19T13:14:59.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.973+0000 7fefd7fff640 1 -- 172.21.15.103:0/14083251 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fefe805d110 con 0x7feff01406f0 2024-08-19T13:14:59.974 INFO:teuthology.orchestra.run.smithi103.stdout:55834574850 2024-08-19T13:14:59.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.976+0000 7f8279c63640 1 -- 172.21.15.103:0/1844425570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f826c0380b0 con 0x7f826c02b3d0 2024-08-19T13:14:59.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.976+0000 7fefd5ffb640 1 -- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fefdc06bf00 msgr2=0x7fefdc06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.976+0000 7fefd5ffb640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fefdc06bf00 0x7fefdc06e3c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fefe00097a0 tx=0x7fefe002e000 comp rx=0 tx=0).stop 2024-08-19T13:14:59.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.977+0000 7f82717fa640 1 -- 172.21.15.103:0/1844425570 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f82740f59a0 con 0x7f826c02b3d0 2024-08-19T13:14:59.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.977+0000 7fefd5ffb640 1 -- 172.21.15.103:0/14083251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff01406f0 msgr2=0x7feff013f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.978 INFO:teuthology.orchestra.run.smithi103.stdout:55834574850 2024-08-19T13:14:59.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.977+0000 7fefd5ffb640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff01406f0 0x7feff013f600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fefe800cad0 tx=0x7fefe800cf90 comp rx=0 tx=0).stop 2024-08-19T13:14:59.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.977+0000 7fefd5ffb640 1 -- 172.21.15.103:0/14083251 shutdown_connections 2024-08-19T13:14:59.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.977+0000 7fefd5ffb640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fefdc06bf00 0x7fefdc06e3c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.977+0000 7fefd5ffb640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff01406f0 0x7feff013f600 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.977+0000 7fefd5ffb640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feff00ae450 0x7feff01470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.977+0000 7fefd5ffb640 1 --2- 172.21.15.103:0/14083251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feff00ac920 0x7feff0144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.977+0000 7fefd5ffb640 1 -- 172.21.15.103:0/14083251 >> 172.21.15.103:0/14083251 conn(0x7feff00b0b80 msgr2=0x7feff014d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.978+0000 7fefd5ffb640 1 -- 172.21.15.103:0/14083251 shutdown_connections 2024-08-19T13:14:59.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.978+0000 7fefd5ffb640 1 -- 172.21.15.103:0/14083251 wait complete. 2024-08-19T13:14:59.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.978+0000 7fc7b3fff640 1 -- 172.21.15.103:0/2362843535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc7b4067ec0 con 0x7fc7b40aa4e0 2024-08-19T13:14:59.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.979+0000 7fc79b7fe640 1 -- 172.21.15.103:0/2362843535 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc7a40590f0 con 0x7fc7b40aa4e0 2024-08-19T13:14:59.980 INFO:teuthology.orchestra.run.smithi103.stdout:55834574850 2024-08-19T13:14:59.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.980+0000 7f825effd640 1 -- 172.21.15.103:0/1844425570 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f825806bf00 msgr2=0x7f825806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.980+0000 7f825effd640 1 --2- 172.21.15.103:0/1844425570 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f825806bf00 0x7f825806e3c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8268006fd0 tx=0x7f8268008040 comp rx=0 tx=0).stop 2024-08-19T13:14:59.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.980+0000 7f825effd640 1 -- 172.21.15.103:0/1844425570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f826c02b3d0 msgr2=0x7f826c02cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.981+0000 7f825effd640 1 --2- 172.21.15.103:0/1844425570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f826c02b3d0 0x7f826c02cc10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f827406b450 tx=0x7f827406a7b0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.981+0000 7f825effd640 1 -- 172.21.15.103:0/1844425570 shutdown_connections 2024-08-19T13:14:59.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.981+0000 7f825effd640 1 --2- 172.21.15.103:0/1844425570 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f825806bf00 0x7f825806e3c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.981+0000 7f825effd640 1 --2- 172.21.15.103:0/1844425570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f826c0c0940 0x7f826c02d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.981+0000 7f825effd640 1 --2- 172.21.15.103:0/1844425570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f826c02b3d0 0x7f826c02cc10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.981+0000 7f825effd640 1 --2- 172.21.15.103:0/1844425570 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f826c02afb0 0x7f826c0b8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.981+0000 7f825effd640 1 -- 172.21.15.103:0/1844425570 >> 172.21.15.103:0/1844425570 conn(0x7f826c005a00 msgr2=0x7f826c0374f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.981+0000 7f825effd640 1 -- 172.21.15.103:0/1844425570 shutdown_connections 2024-08-19T13:14:59.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.981+0000 7f825effd640 1 -- 172.21.15.103:0/1844425570 wait complete. 2024-08-19T13:14:59.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.982+0000 7fc7997fa640 1 -- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc7a006bf30 msgr2=0x7fc7a006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.982+0000 7fc7997fa640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc7a006bf30 0x7fc7a006e3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac035000 tx=0x7fc7ac00b030 comp rx=0 tx=0).stop 2024-08-19T13:14:59.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.983+0000 7fc7997fa640 1 -- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b40aa4e0 msgr2=0x7fc7b40aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.983+0000 7fad7bf76640 1 -- 172.21.15.103:0/1743776220 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fad74075c30 con 0x7fad7412b650 2024-08-19T13:14:59.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.984+0000 7fad727fc640 1 -- 172.21.15.103:0/1743776220 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fad6805c870 con 0x7fad7412b650 2024-08-19T13:14:59.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.984+0000 7fc7997fa640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b40aa4e0 0x7fc7b40aa8c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400eb40 tx=0x7fc7a4005e10 comp rx=0 tx=0).stop 2024-08-19T13:14:59.985 INFO:teuthology.orchestra.run.smithi103.stdout:55834574851 2024-08-19T13:14:59.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.984+0000 7fc7997fa640 1 -- 172.21.15.103:0/2362843535 shutdown_connections 2024-08-19T13:14:59.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.984+0000 7fc7997fa640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc7a006bf30 0x7fc7a006e3f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.984+0000 7fc7997fa640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b41db9b0 0x7fc7b41dbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.984+0000 7fc7997fa640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7b413e790 0x7fc7b413ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.984+0000 7fc7997fa640 1 --2- 172.21.15.103:0/2362843535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b40aa4e0 0x7fc7b40aa8c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.984+0000 7fc7997fa640 1 -- 172.21.15.103:0/2362843535 >> 172.21.15.103:0/2362843535 conn(0x7fc7b40b3eb0 msgr2=0x7fc7b40b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.985+0000 7fc7997fa640 1 -- 172.21.15.103:0/2362843535 shutdown_connections 2024-08-19T13:14:59.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.985+0000 7fc7997fa640 1 -- 172.21.15.103:0/2362843535 wait complete. 2024-08-19T13:14:59.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.986+0000 7fcf0ffff640 1 -- 172.21.15.103:0/3795937062 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcf08005640 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.988 INFO:teuthology.orchestra.run.smithi103.stdout:55834574850 2024-08-19T13:14:59.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.987+0000 7fcf227fc640 1 -- 172.21.15.103:0/3795937062 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf1c00e280 con 0x7fcf2c13d9d0 2024-08-19T13:14:59.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.988+0000 7fad7bf76640 1 -- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fad6006bf00 msgr2=0x7fad6006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.988+0000 7fad7bf76640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fad6006bf00 0x7fad6006e3c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fad64009c40 tx=0x7fad64008040 comp rx=0 tx=0).stop 2024-08-19T13:14:59.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.988+0000 7fad7bf76640 1 -- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad7412b650 msgr2=0x7fad740a9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.988+0000 7fad7bf76640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad7412b650 0x7fad740a9ba0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fad6800c760 tx=0x7fad6800cc20 comp rx=0 tx=0).stop 2024-08-19T13:14:59.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.988+0000 7fad7bf76640 1 -- 172.21.15.103:0/1743776220 shutdown_connections 2024-08-19T13:14:59.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.988+0000 7fad7bf76640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fad6006bf00 0x7fad6006e3c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.988+0000 7fad7bf76640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad741412a0 0x7fad74141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.988+0000 7fad7bf76640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad7412b650 0x7fad740a9ba0 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fad6800c760 tx=0x7fad6800cc20 comp rx=0 tx=0).stop 2024-08-19T13:14:59.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.988+0000 7fad7bf76640 1 --2- 172.21.15.103:0/1743776220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c0051c0 0x7fad7413eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.989+0000 7fad7bf76640 1 -- 172.21.15.103:0/1743776220 >> 172.21.15.103:0/1743776220 conn(0x7fad740b41c0 msgr2=0x7fad740b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.989+0000 7fad7bf76640 1 -- 172.21.15.103:0/1743776220 shutdown_connections 2024-08-19T13:14:59.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.989+0000 7fad7bf76640 1 -- 172.21.15.103:0/1743776220 wait complete. 2024-08-19T13:14:59.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.991+0000 7fcf0ffff640 1 -- 172.21.15.103:0/3795937062 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fcf1006bf00 msgr2=0x7fcf1006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.991+0000 7fcf0ffff640 1 --2- 172.21.15.103:0/3795937062 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fcf1006bf00 0x7fcf1006e3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcf24007fd0 tx=0x7fcf24009040 comp rx=0 tx=0).stop 2024-08-19T13:14:59.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.991+0000 7fcf0ffff640 1 -- 172.21.15.103:0/3795937062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf2c13d9d0 msgr2=0x7fcf2c0b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:14:59.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.991+0000 7fcf0ffff640 1 --2- 172.21.15.103:0/3795937062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf2c13d9d0 0x7fcf2c0b1bd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c001eb0 tx=0x7fcf1c00c870 comp rx=0 tx=0).stop 2024-08-19T13:14:59.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.991+0000 7fcf0ffff640 1 -- 172.21.15.103:0/3795937062 shutdown_connections 2024-08-19T13:14:59.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.991+0000 7fcf0ffff640 1 --2- 172.21.15.103:0/3795937062 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fcf1006bf00 0x7fcf1006e3c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.991+0000 7fcf0ffff640 1 --2- 172.21.15.103:0/3795937062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf2c13d9d0 0x7fcf2c0b1bd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.991+0000 7fcf0ffff640 1 --2- 172.21.15.103:0/3795937062 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf2c071540 0x7fcf2c135950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.991+0000 7fcf0ffff640 1 --2- 172.21.15.103:0/3795937062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf2c071160 0x7fcf2c135110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:14:59.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.991+0000 7fcf0ffff640 1 -- 172.21.15.103:0/3795937062 >> 172.21.15.103:0/3795937062 conn(0x7fcf2c0b0c60 msgr2=0x7fcf2c0af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:14:59.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.992+0000 7fcf0ffff640 1 -- 172.21.15.103:0/3795937062 shutdown_connections 2024-08-19T13:14:59.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.992+0000 7fcf0ffff640 1 -- 172.21.15.103:0/3795937062 wait complete. 2024-08-19T13:14:59.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.995+0000 7fec0372c640 1 -- 172.21.15.103:0/3720851542 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7febfc072c50 con 0x7febfc14ef20 2024-08-19T13:14:59.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.996+0000 7f3229a2d640 1 -- 172.21.15.103:0/1944317711 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f321c0332d0 con 0x7f321c0c0af0 2024-08-19T13:14:59.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.996+0000 7febf1ffb640 1 -- 172.21.15.103:0/3720851542 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7febec020020 con 0x7febfc14ef20 2024-08-19T13:14:59.997 INFO:teuthology.orchestra.run.smithi103.stdout:55834574850 2024-08-19T13:14:59.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.0 2024-08-19T13:14:59.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.997+0000 7f107d7fa640 1 -- 172.21.15.103:0/3786692694 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f10980008d0 con 0x7f1098072f60 2024-08-19T13:14:59.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.997+0000 7f32217fa640 1 -- 172.21.15.103:0/1944317711 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f321805eb90 con 0x7f321c0c0af0 2024-08-19T13:14:59.999 INFO:teuthology.orchestra.run.smithi103.stdout:55834574849 2024-08-19T13:14:59.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:14:59.997+0000 7f107f7fe640 1 -- 172.21.15.103:0/3786692694 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f108805d620 con 0x7f1098072f60 2024-08-19T13:14:59.999 INFO:teuthology.orchestra.run.smithi103.stdout:55834574850 2024-08-19T13:15:00.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.000+0000 7fec0372c640 1 -- 172.21.15.103:0/3720851542 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7febe006c230 msgr2=0x7febe006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:00.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.000+0000 7fec0372c640 1 --2- 172.21.15.103:0/3720851542 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7febe006c230 0x7febe006e6f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7febf80014e0 tx=0x7febf80079d0 comp rx=0 tx=0).stop 2024-08-19T13:15:00.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.000+0000 7fec0372c640 1 -- 172.21.15.103:0/3720851542 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7febfc14ef20 msgr2=0x7febfc151870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:00.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.000+0000 7fec0372c640 1 --2- 172.21.15.103:0/3720851542 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7febfc14ef20 0x7febfc151870 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7febec00c550 tx=0x7febec00ca10 comp rx=0 tx=0).stop 2024-08-19T13:15:00.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.000+0000 7fec0372c640 1 -- 172.21.15.103:0/3720851542 shutdown_connections 2024-08-19T13:15:00.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.000+0000 7fec0372c640 1 --2- 172.21.15.103:0/3720851542 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7febe006c230 0x7febe006e6f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:00.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7fec0372c640 1 --2- 172.21.15.103:0/3720851542 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7febfc14ef20 0x7febfc151870 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:00.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7fec0372c640 1 --2- 172.21.15.103:0/3720851542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febfc14f450 0x7febfc151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:00.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7fec0372c640 1 --2- 172.21.15.103:0/3720851542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febfc072410 0x7febfc14d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:00.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7fec0372c640 1 -- 172.21.15.103:0/3720851542 >> 172.21.15.103:0/3720851542 conn(0x7febfc0b80d0 msgr2=0x7febfc0b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:00.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7fec0372c640 1 -- 172.21.15.103:0/3720851542 shutdown_connections 2024-08-19T13:15:00.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7fec0372c640 1 -- 172.21.15.103:0/3720851542 wait complete. 2024-08-19T13:15:00.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7f3229a2d640 1 -- 172.21.15.103:0/1944317711 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f321006bf00 msgr2=0x7f321006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:00.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7f3229a2d640 1 --2- 172.21.15.103:0/1944317711 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f321006bf00 0x7f321006e3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3214000f60 tx=0x7f3214007d30 comp rx=0 tx=0).stop 2024-08-19T13:15:00.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7f3229a2d640 1 -- 172.21.15.103:0/1944317711 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f321c0c0af0 msgr2=0x7f321c02f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:00.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7f3229a2d640 1 --2- 172.21.15.103:0/1944317711 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f321c0c0af0 0x7f321c02f700 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f321800c3c0 tx=0x7f321800c880 comp rx=0 tx=0).stop 2024-08-19T13:15:00.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7f3229a2d640 1 -- 172.21.15.103:0/1944317711 shutdown_connections 2024-08-19T13:15:00.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7f3229a2d640 1 --2- 172.21.15.103:0/1944317711 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f321006bf00 0x7f321006e3c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:00.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7f3229a2d640 1 --2- 172.21.15.103:0/1944317711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c0c4610 0x7f321c0bc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:00.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7f3229a2d640 1 --2- 172.21.15.103:0/1944317711 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f321c0c0af0 0x7f321c02f700 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:00.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7f3229a2d640 1 --2- 172.21.15.103:0/1944317711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f321c034f90 0x7f321c030280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:00.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7f3229a2d640 1 -- 172.21.15.103:0/1944317711 >> 172.21.15.103:0/1944317711 conn(0x7f321c0066a0 msgr2=0x7f321c034460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:00.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7f3229a2d640 1 -- 172.21.15.103:0/1944317711 shutdown_connections 2024-08-19T13:15:00.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7f3229a2d640 1 -- 172.21.15.103:0/1944317711 wait complete. 2024-08-19T13:15:00.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.2 2024-08-19T13:15:00.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7f107d7fa640 1 -- 172.21.15.103:0/3786692694 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f108406c260 msgr2=0x7f108406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:00.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.001+0000 7f107d7fa640 1 --2- 172.21.15.103:0/3786692694 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f108406c260 0x7f108406e720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f108c037000 tx=0x7f108c032000 comp rx=0 tx=0).stop 2024-08-19T13:15:00.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.002+0000 7f107d7fa640 1 -- 172.21.15.103:0/3786692694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1098072f60 msgr2=0x7f1098140ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:00.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.002+0000 7f107d7fa640 1 --2- 172.21.15.103:0/3786692694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1098072f60 0x7f1098140ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f108800ebe0 tx=0x7f108800c520 comp rx=0 tx=0).stop 2024-08-19T13:15:00.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.002+0000 7f107d7fa640 1 -- 172.21.15.103:0/3786692694 shutdown_connections 2024-08-19T13:15:00.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.002+0000 7f107d7fa640 1 --2- 172.21.15.103:0/3786692694 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f108406c260 0x7f108406e720 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:00.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.002+0000 7f107d7fa640 1 --2- 172.21.15.103:0/3786692694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f109813cb20 0x7f10981e3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:00.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.002+0000 7f107d7fa640 1 --2- 172.21.15.103:0/3786692694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10980aec80 0x7f1098143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:00.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.002+0000 7f107d7fa640 1 --2- 172.21.15.103:0/3786692694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1098072f60 0x7f1098140ae0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f108800ebe0 tx=0x7f108800c520 comp rx=0 tx=0).stop 2024-08-19T13:15:00.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.002+0000 7f107d7fa640 1 -- 172.21.15.103:0/3786692694 >> 172.21.15.103:0/3786692694 conn(0x7f10980b41c0 msgr2=0x7f1098135590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:00.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.002+0000 7f107d7fa640 1 -- 172.21.15.103:0/3786692694 shutdown_connections 2024-08-19T13:15:00.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:00.002+0000 7f107d7fa640 1 -- 172.21.15.103:0/3786692694 wait complete. 2024-08-19T13:15:00.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.6 2024-08-19T13:15:00.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.4 2024-08-19T13:15:00.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.1 2024-08-19T13:15:00.017 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.5 2024-08-19T13:15:00.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574849 for osd.7 2024-08-19T13:15:00.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.3 2024-08-19T13:15:00.998 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-19T13:15:01.005 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-19T13:15:01.009 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-08-19T13:15:01.013 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-19T13:15:01.013 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-19T13:15:01.018 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-19T13:15:01.019 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-08-19T13:15:01.020 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-19T13:15:01.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.157+0000 7f3f5474b640 1 Processor -- start 2024-08-19T13:15:01.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.157+0000 7f3f5474b640 1 -- start start 2024-08-19T13:15:01.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.158+0000 7f3f5474b640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f4c072560 0x7f3f4c0729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.158+0000 7f3f5474b640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f4c0739f0 con 0x7f3f4c06f500 2024-08-19T13:15:01.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.158+0000 7f3f5474b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f4c06f8e0 con 0x7f3f4c070330 2024-08-19T13:15:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.158+0000 7f3f5474b640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4c071470 con 0x7f3f4c072560 2024-08-19T13:15:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.158+0000 7f3f524e7640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f3f4c06f500 0x7f3f4c0ae7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:51150/0 (socket says 172.21.15.103:51150) 2024-08-19T13:15:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.159+0000 7f3f524e7640 1 -- 172.21.15.103:0/937620392 learned_addr learned my addr 172.21.15.103:0/937620392 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:15:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.159+0000 7f3f52ce8640 1 --2- 172.21.15.103:0/937620392 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f4c072560 0x7f3f4c0729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.159+0000 7f3f52ce8640 1 -- 172.21.15.103:0/937620392 >> v1:172.21.15.122:6789/0 conn(0x7f3f4c070330 legacy=0x7f3f4c071e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.159+0000 7f3f52ce8640 1 -- 172.21.15.103:0/937620392 >> v1:172.21.15.103:6789/0 conn(0x7f3f4c06f500 legacy=0x7f3f4c0ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.159+0000 7f3f52ce8640 1 -- 172.21.15.103:0/937620392 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4c071750 con 0x7f3f4c072560 2024-08-19T13:15:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.161+0000 7f3f52ce8640 1 --2- 172.21.15.103:0/937620392 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f4c072560 0x7f3f4c0729d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3f440097a0 tx=0x7f3f4402eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f7dd75e7f96822 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.162+0000 7f3f514e5640 1 -- 172.21.15.103:0/937620392 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44036070 con 0x7f3f4c072560 2024-08-19T13:15:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.162+0000 7f3f514e5640 1 -- 172.21.15.103:0/937620392 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f44009390 con 0x7f3f4c072560 2024-08-19T13:15:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.163+0000 7f3f5474b640 1 -- 172.21.15.103:0/937620392 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f4c072560 msgr2=0x7f3f4c0729d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.163+0000 7f3f5474b640 1 --2- 172.21.15.103:0/937620392 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f4c072560 0x7f3f4c0729d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3f440097a0 tx=0x7f3f4402eb90 comp rx=0 tx=0).stop 2024-08-19T13:15:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.164+0000 7f3f5474b640 1 -- 172.21.15.103:0/937620392 shutdown_connections 2024-08-19T13:15:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.164+0000 7f3f5474b640 1 --2- 172.21.15.103:0/937620392 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f4c072560 0x7f3f4c0729d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.164+0000 7f3f5474b640 1 -- 172.21.15.103:0/937620392 >> 172.21.15.103:0/937620392 conn(0x7f3f4c0b3fb0 msgr2=0x7f3f4c0b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.164+0000 7f3f5474b640 1 -- 172.21.15.103:0/937620392 shutdown_connections 2024-08-19T13:15:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.164+0000 7f3f5474b640 1 -- 172.21.15.103:0/937620392 wait complete. 2024-08-19T13:15:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.166+0000 7f3f5474b640 1 Processor -- start 2024-08-19T13:15:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.166+0000 7f3f5474b640 1 -- start start 2024-08-19T13:15:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.166+0000 7f3f5474b640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f4c06f500 0x7f3f4c0ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.167+0000 7f3f5474b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f4c070330 0x7f3f4c0ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.167+0000 7f3f5474b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f4c072560 0x7f3f4c0ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.167+0000 7f3f5474b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c072f10 con 0x7f3f4c072560 2024-08-19T13:15:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.167+0000 7f3f5474b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c06fc20 con 0x7f3f4c070330 2024-08-19T13:15:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.167+0000 7f3f5474b640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4c15abe0 con 0x7f3f4c06f500 2024-08-19T13:15:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.167+0000 7f3f51ce6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f4c070330 0x7f3f4c0ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.167+0000 7f3f51ce6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f4c070330 0x7f3f4c0ad290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.103:48808/0 (socket says 172.21.15.103:48808) 2024-08-19T13:15:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.167+0000 7f3f51ce6640 1 -- 172.21.15.103:0/3898367543 learned_addr learned my addr 172.21.15.103:0/3898367543 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.167+0000 7f3f524e7640 1 --2- 172.21.15.103:0/3898367543 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f4c06f500 0x7f3f4c0ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.168+0000 7f3f51ce6640 1 -- 172.21.15.103:0/3898367543 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f4c06f500 msgr2=0x7f3f4c0ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.168+0000 7f3f51ce6640 1 --2- 172.21.15.103:0/3898367543 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f4c06f500 0x7f3f4c0ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.168+0000 7f3f51ce6640 1 -- 172.21.15.103:0/3898367543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f4c072560 msgr2=0x7f3f4c0ad7d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T13:15:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.168+0000 7f3f51ce6640 1 --2- 172.21.15.103:0/3898367543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f4c072560 0x7f3f4c0ad7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.168+0000 7f3f51ce6640 1 -- 172.21.15.103:0/3898367543 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4c153d20 con 0x7f3f4c070330 2024-08-19T13:15:01.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.168+0000 7f3f524e7640 1 --2- 172.21.15.103:0/3898367543 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f4c06f500 0x7f3f4c0ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:15:01.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.168+0000 7f3f51ce6640 1 --2- 172.21.15.103:0/3898367543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f4c070330 0x7f3f4c0ad290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00c550 tx=0x7f3f3c00ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.169+0000 7f3f42ffd640 1 -- 172.21.15.103:0/3898367543 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c005480 con 0x7f3f4c070330 2024-08-19T13:15:01.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.170+0000 7f3f5474b640 1 -- 172.21.15.103:0/3898367543 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4c153f90 con 0x7f3f4c070330 2024-08-19T13:15:01.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.170+0000 7f3f5474b640 1 -- 172.21.15.103:0/3898367543 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f4c154550 con 0x7f3f4c070330 2024-08-19T13:15:01.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.171+0000 7f3f42ffd640 1 -- 172.21.15.103:0/3898367543 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3c0035f0 con 0x7f3f4c070330 2024-08-19T13:15:01.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.171+0000 7f3f42ffd640 1 -- 172.21.15.103:0/3898367543 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0039e0 con 0x7f3f4c070330 2024-08-19T13:15:01.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.173+0000 7f3f5474b640 1 -- 172.21.15.103:0/3898367543 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f4c0729d0 con 0x7f3f4c070330 2024-08-19T13:15:01.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.174+0000 7f3f42ffd640 1 -- 172.21.15.103:0/3898367543 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f3c003b80 con 0x7f3f4c070330 2024-08-19T13:15:01.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.175+0000 7f3f42ffd640 1 --2- 172.21.15.103:0/3898367543 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f3f3006c0a0 0x7f3f3006e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.175+0000 7f3f524e7640 1 --2- 172.21.15.103:0/3898367543 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f3f3006c0a0 0x7f3f3006e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.176+0000 7f3f524e7640 1 --2- 172.21.15.103:0/3898367543 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f3f3006c0a0 0x7f3f3006e560 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3f48000f60 tx=0x7f3f48000bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.176+0000 7f3f42ffd640 1 -- 172.21.15.103:0/3898367543 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f3f3c08ab70 con 0x7f3f4c070330 2024-08-19T13:15:01.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.178+0000 7f3f42ffd640 1 -- 172.21.15.103:0/3898367543 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f3f3c059740 con 0x7f3f4c070330 2024-08-19T13:15:01.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.216+0000 7f2f1acb5640 1 Processor -- start 2024-08-19T13:15:01.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.216+0000 7f2f1acb5640 1 -- start start 2024-08-19T13:15:01.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.217+0000 7f2f1acb5640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f2f1406f290 0x7f2f14071e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.217+0000 7f2f1acb5640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2f1406f700 0x7f2f14072390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.217+0000 7f2f1acb5640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f14071470 con 0x7f2f14070330 2024-08-19T13:15:01.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.217+0000 7f2f1acb5640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f2f14071750 con 0x7f2f1406f290 2024-08-19T13:15:01.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.217+0000 7f2f1acb5640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f2f14071a30 con 0x7f2f1406f700 2024-08-19T13:15:01.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.217+0000 7f2f13fff640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f2f1406f290 0x7f2f14071e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.217+0000 7f2f13fff640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f2f1406f290 0x7f2f14071e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49974/0 (socket says 172.21.15.103:49974) 2024-08-19T13:15:01.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.217+0000 7f2f13fff640 1 -- 172.21.15.103:0/1910892147 learned_addr learned my addr 172.21.15.103:0/1910892147 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.217+0000 7f2f19252640 1 --2- 172.21.15.103:0/1910892147 >> v2:172.21.15.122:3300/0 conn(0x7f2f1406f700 0x7f2f14072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.218+0000 7f2f137fe640 1 -- 172.21.15.103:0/1910892147 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 712778960 0 0) 0x7f2f14071470 con 0x7f2f14070330 2024-08-19T13:15:01.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.218+0000 7f2f137fe640 1 -- 172.21.15.103:0/1910892147 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2efc003520 con 0x7f2f14070330 2024-08-19T13:15:01.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.218+0000 7f2f19252640 1 -- 172.21.15.103:0/1910892147 >> v2:172.21.15.103:3300/0 conn(0x7f2f1406f290 msgr2=0x7f2f14071e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:15:01.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.218+0000 7f2f19252640 1 --2- 172.21.15.103:0/1910892147 >> v2:172.21.15.103:3300/0 conn(0x7f2f1406f290 0x7f2f14071e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.218+0000 7f2f19252640 1 -- 172.21.15.103:0/1910892147 >> v1:172.21.15.122:6789/0 conn(0x7f2f14070330 legacy=0x7f2f140ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.218+0000 7f2f19252640 1 -- 172.21.15.103:0/1910892147 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1406fc20 con 0x7f2f1406f700 2024-08-19T13:15:01.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.219+0000 7f2f19252640 1 --2- 172.21.15.103:0/1910892147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f1406f700 0x7f2f14072390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2f080097a0 tx=0x7f2f0803b160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e690d912c8a49b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.219+0000 7ff58124e640 1 Processor -- start 2024-08-19T13:15:01.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.219+0000 7ff58124e640 1 -- start start 2024-08-19T13:15:01.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.219+0000 7f2f137fe640 1 -- 172.21.15.103:0/1910892147 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08048070 con 0x7f2f1406f700 2024-08-19T13:15:01.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.219+0000 7f2f137fe640 1 -- 172.21.15.103:0/1910892147 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f08043440 con 0x7f2f1406f700 2024-08-19T13:15:01.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.219+0000 7f2f137fe640 1 -- 172.21.15.103:0/1910892147 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08043810 con 0x7f2f1406f700 2024-08-19T13:15:01.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.220+0000 7fd21d05c640 1 Processor -- start 2024-08-19T13:15:01.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.220+0000 7f1427fff640 1 Processor -- start 2024-08-19T13:15:01.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.220+0000 7f1427fff640 1 -- start start 2024-08-19T13:15:01.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7f1427fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1420037110 0x7f1420035590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7f1427fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14200360d0 0x7f14200cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7f1427fff640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f14200cbef0 0x7f14200c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7f1427fff640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f14200374f0 con 0x7f14200cbef0 2024-08-19T13:15:01.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7f1427fff640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f1420035c40 con 0x7f1420037110 2024-08-19T13:15:01.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7f1427fff640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f142002beb0 con 0x7f14200360d0 2024-08-19T13:15:01.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7f1426ffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1420037110 0x7f1420035590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7f1426ffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1420037110 0x7f1420035590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.103:48820/0 (socket says 172.21.15.103:48820) 2024-08-19T13:15:01.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7f1426ffd640 1 -- 172.21.15.103:0/2778699009 learned_addr learned my addr 172.21.15.103:0/2778699009 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.220+0000 7f2f1acb5640 1 -- 172.21.15.103:0/1910892147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f1406f700 msgr2=0x7f2f14072390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.220+0000 7f2f1acb5640 1 --2- 172.21.15.103:0/1910892147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f1406f700 0x7f2f14072390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2f080097a0 tx=0x7f2f0803b160 comp rx=0 tx=0).stop 2024-08-19T13:15:01.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.220+0000 7f2f1acb5640 1 -- 172.21.15.103:0/1910892147 shutdown_connections 2024-08-19T13:15:01.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.220+0000 7f2f1acb5640 1 --2- 172.21.15.103:0/1910892147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f1406f700 0x7f2f14072390 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.220+0000 7f2f1acb5640 1 --2- 172.21.15.103:0/1910892147 >> v2:172.21.15.103:3300/0 conn(0x7f2f1406f290 0x7f2f14071e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.220+0000 7f2f1acb5640 1 -- 172.21.15.103:0/1910892147 >> 172.21.15.103:0/1910892147 conn(0x7f2f140b3fb0 msgr2=0x7f2f140b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7f2f1acb5640 1 -- 172.21.15.103:0/1910892147 shutdown_connections 2024-08-19T13:15:01.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7f2f1acb5640 1 -- 172.21.15.103:0/1910892147 wait complete. 2024-08-19T13:15:01.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.220+0000 7ff58124e640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff57c06c590 0x7ff57c06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.220+0000 7ff58124e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff57c070440 0x7ff57c071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.220+0000 7ff58124e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff57c0715d0 con 0x7ff57c141dc0 2024-08-19T13:15:01.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.220+0000 7ff58124e640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7ff57c06c970 con 0x7ff57c06c590 2024-08-19T13:15:01.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.220+0000 7ff58124e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7ff57c06fc10 con 0x7ff57c070440 2024-08-19T13:15:01.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7ff57a59c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff57c070440 0x7ff57c071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7ff57a59c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff57c070440 0x7ff57c071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.103:48816/0 (socket says 172.21.15.103:48816) 2024-08-19T13:15:01.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7ff57a59c640 1 -- 172.21.15.103:0/741574466 learned_addr learned my addr 172.21.15.103:0/741574466 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7ff57ad9d640 1 --2- 172.21.15.103:0/741574466 >> v2:172.21.15.103:3300/0 conn(0x7ff57c06c590 0x7ff57c06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7fd21d05c640 1 -- start start 2024-08-19T13:15:01.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7fd21d05c640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fd2100c4800 0x7fd2100c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7fd21d05c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd2100caa10 0x7fd2100cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7fd21d05c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2100304d0 con 0x7fd210038b00 2024-08-19T13:15:01.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7fd21d05c640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fd210038ee0 con 0x7fd2100c4800 2024-08-19T13:15:01.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7fd21d05c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fd2100cb390 con 0x7fd2100caa10 2024-08-19T13:15:01.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.221+0000 7f14277fe640 1 --2- 172.21.15.103:0/2778699009 >> v2:172.21.15.103:3300/0 conn(0x7f14200cbef0 0x7f14200c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7f14267fc640 1 --2- 172.21.15.103:0/2778699009 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14200360d0 0x7f14200cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7ff57a59c640 1 -- 172.21.15.103:0/741574466 >> v2:172.21.15.103:3300/0 conn(0x7ff57c06c590 msgr2=0x7ff57c06f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:15:01.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7ff57a59c640 1 --2- 172.21.15.103:0/741574466 >> v2:172.21.15.103:3300/0 conn(0x7ff57c06c590 0x7ff57c06f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7ff57a59c640 1 -- 172.21.15.103:0/741574466 >> v1:172.21.15.122:6789/0 conn(0x7ff57c141dc0 legacy=0x7ff57c142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7ff57a59c640 1 -- 172.21.15.103:0/741574466 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57c0718b0 con 0x7ff57c070440 2024-08-19T13:15:01.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7f1426ffd640 1 -- 172.21.15.103:0/2778699009 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14200360d0 msgr2=0x7f14200cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:15:01.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7f1426ffd640 1 --2- 172.21.15.103:0/2778699009 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14200360d0 0x7f14200cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7f1426ffd640 1 -- 172.21.15.103:0/2778699009 >> v2:172.21.15.103:3300/0 conn(0x7f14200cbef0 msgr2=0x7f14200c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:15:01.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7f1426ffd640 1 --2- 172.21.15.103:0/2778699009 >> v2:172.21.15.103:3300/0 conn(0x7f14200cbef0 0x7f14200c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7f1426ffd640 1 -- 172.21.15.103:0/2778699009 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142002c120 con 0x7f1420037110 2024-08-19T13:15:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7fd217fff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fd210038b00 0x7fd2100c74b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.103:60098/0 (socket says 172.21.15.103:60098) 2024-08-19T13:15:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7fd2177fe640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fd2100c4800 0x7fd2100c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7fd2177fe640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fd2100c4800 0x7fd2100c4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49994/0 (socket says 172.21.15.103:49994) 2024-08-19T13:15:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7fd2177fe640 1 -- 172.21.15.103:0/2487406249 learned_addr learned my addr 172.21.15.103:0/2487406249 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7fd21c85b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd2100caa10 0x7fd2100cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7f2f1acb5640 1 Processor -- start 2024-08-19T13:15:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7fd216ffd640 1 -- 172.21.15.103:0/2487406249 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3606598601 0 0) 0x7fd2100304d0 con 0x7fd210038b00 2024-08-19T13:15:01.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7f14277fe640 1 --2- 172.21.15.103:0/2778699009 >> v2:172.21.15.103:3300/0 conn(0x7f14200cbef0 0x7f14200c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:15:01.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.222+0000 7f2f1acb5640 1 -- start start 2024-08-19T13:15:01.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7fd2177fe640 1 -- 172.21.15.103:0/2487406249 >> v2:172.21.15.122:3300/0 conn(0x7fd2100caa10 msgr2=0x7fd2100cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:15:01.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7fd2177fe640 1 --2- 172.21.15.103:0/2487406249 >> v2:172.21.15.122:3300/0 conn(0x7fd2100caa10 0x7fd2100cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7fd2177fe640 1 -- 172.21.15.103:0/2487406249 >> v1:172.21.15.122:6789/0 conn(0x7fd210038b00 legacy=0x7fd2100c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7fd2177fe640 1 -- 172.21.15.103:0/2487406249 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd210030740 con 0x7fd2100c4800 2024-08-19T13:15:01.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7ff57a59c640 1 --2- 172.21.15.103:0/741574466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c070440 0x7ff57c071d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0adc50 tx=0x7ff57002c780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8c9ef2e5fc74d64 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7f2f1acb5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f1406f290 0x7f2f140adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7f2f1acb5640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2f14070330 0x7f2f1408a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7ff579d9b640 1 -- 172.21.15.103:0/741574466 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570032070 con 0x7ff57c070440 2024-08-19T13:15:01.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7ff579d9b640 1 -- 172.21.15.103:0/741574466 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff570032620 con 0x7ff57c070440 2024-08-19T13:15:01.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7f2f1acb5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1413d9c0 0x7f2f1408a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7f2f1acb5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f14072c60 con 0x7f2f1413d9c0 2024-08-19T13:15:01.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7f2f1acb5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f14072ae0 con 0x7f2f1406f290 2024-08-19T13:15:01.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7f2f1acb5640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f2f14072de0 con 0x7f2f14070330 2024-08-19T13:15:01.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7ff579d9b640 1 -- 172.21.15.103:0/741574466 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5700329f0 con 0x7ff57c070440 2024-08-19T13:15:01.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7f2f18a51640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f1406f290 0x7f2f140adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7f1426ffd640 1 --2- 172.21.15.103:0/2778699009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1420037110 0x7f1420035590 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1428069ce0 tx=0x7f14280a53a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3810d78dd32cb5db server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7f2f18a51640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f1406f290 0x7f2f140adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.103:48850/0 (socket says 172.21.15.103:48850) 2024-08-19T13:15:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.223+0000 7f2f18a51640 1 -- 172.21.15.103:0/3464848685 learned_addr learned my addr 172.21.15.103:0/3464848685 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7fc4cad7a640 1 Processor -- start 2024-08-19T13:15:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7f2f13fff640 1 --2- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2f14070330 0x7f2f1408a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7f2f19252640 1 --2- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1413d9c0 0x7f2f1408a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7ff58124e640 1 -- 172.21.15.103:0/741574466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c070440 msgr2=0x7ff57c071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7ff58124e640 1 --2- 172.21.15.103:0/741574466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c070440 0x7ff57c071d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0adc50 tx=0x7ff57002c780 comp rx=0 tx=0).stop 2024-08-19T13:15:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7fc4cad7a640 1 -- start start 2024-08-19T13:15:01.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7f2f13fff640 1 -- 172.21.15.103:0/3464848685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f1406f290 msgr2=0x7f2f140adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7f2f13fff640 1 --2- 172.21.15.103:0/3464848685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f1406f290 0x7f2f140adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7ff58124e640 1 -- 172.21.15.103:0/741574466 shutdown_connections 2024-08-19T13:15:01.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7ff58124e640 1 --2- 172.21.15.103:0/741574466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c070440 0x7ff57c071d40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7ff58124e640 1 --2- 172.21.15.103:0/741574466 >> v2:172.21.15.103:3300/0 conn(0x7ff57c06c590 0x7ff57c06f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7ff58124e640 1 -- 172.21.15.103:0/741574466 >> 172.21.15.103:0/741574466 conn(0x7ff57c06b800 msgr2=0x7ff57c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7fd2177fe640 1 --2- 172.21.15.103:0/2487406249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2100c4800 0x7fd2100c4c70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd210006eb0 tx=0x7fd20c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3ae33b8b50f68f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7f90fb00c640 1 Processor -- start 2024-08-19T13:15:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7f2f13fff640 1 -- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1413d9c0 msgr2=0x7f2f1408a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7f2f13fff640 1 --2- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1413d9c0 0x7f2f1408a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7f2f13fff640 1 -- 172.21.15.103:0/3464848685 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1413ea60 con 0x7f2f14070330 2024-08-19T13:15:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7f1425ffb640 1 -- 172.21.15.103:0/2778699009 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14280b2070 con 0x7f1420037110 2024-08-19T13:15:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.224+0000 7f1425ffb640 1 -- 172.21.15.103:0/2778699009 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14280ad6a0 con 0x7f1420037110 2024-08-19T13:15:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7f1425ffb640 1 -- 172.21.15.103:0/2778699009 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14280adaf0 con 0x7f1420037110 2024-08-19T13:15:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7f1427fff640 1 -- 172.21.15.103:0/2778699009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1420037110 msgr2=0x7f1420035590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7f1427fff640 1 --2- 172.21.15.103:0/2778699009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1420037110 0x7f1420035590 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1428069ce0 tx=0x7f14280a53a0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7ff58124e640 1 -- 172.21.15.103:0/741574466 shutdown_connections 2024-08-19T13:15:01.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7fc4cad7a640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fc4c40b4300 0x7fc4c40b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7fc4cad7a640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4c40b2320 0x7fc4c40adf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7f2f13fff640 1 --2- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2f14070330 0x7f2f1408a480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2f040149a0 tx=0x7f2f04014e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7f2f117fa640 1 -- 172.21.15.103:0/3464848685 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f040129c0 con 0x7f2f14070330 2024-08-19T13:15:01.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7fd216ffd640 1 -- 172.21.15.103:0/2487406249 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c037070 con 0x7fd2100c4800 2024-08-19T13:15:01.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7fd216ffd640 1 -- 172.21.15.103:0/2487406249 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20c032440 con 0x7fd2100c4800 2024-08-19T13:15:01.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7f2f1acb5640 1 -- 172.21.15.103:0/3464848685 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f1413ed40 con 0x7f2f14070330 2024-08-19T13:15:01.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f1427fff640 1 -- 172.21.15.103:0/2778699009 shutdown_connections 2024-08-19T13:15:01.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f1427fff640 1 --2- 172.21.15.103:0/2778699009 >> v2:172.21.15.103:3300/0 conn(0x7f14200cbef0 0x7f14200c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f1427fff640 1 --2- 172.21.15.103:0/2778699009 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14200360d0 0x7f14200cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f1427fff640 1 --2- 172.21.15.103:0/2778699009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1420037110 0x7f1420035590 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f1427fff640 1 -- 172.21.15.103:0/2778699009 >> 172.21.15.103:0/2778699009 conn(0x7f142003a420 msgr2=0x7f1420036680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7f90fb00c640 1 -- start start 2024-08-19T13:15:01.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f90fb00c640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f90f40aa280 0x7f90f4143170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f90fb00c640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90f40afd50 con 0x7f90f40a9ea0 2024-08-19T13:15:01.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f90fb00c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90f40aadd0 con 0x7f90f40ab090 2024-08-19T13:15:01.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f90fb00c640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f90f4142ce0 con 0x7f90f40aa280 2024-08-19T13:15:01.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7fd216ffd640 1 -- 172.21.15.103:0/2487406249 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c032810 con 0x7fd2100c4800 2024-08-19T13:15:01.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7ff58124e640 1 -- 172.21.15.103:0/741574466 wait complete. 2024-08-19T13:15:01.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f90fa00a640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f90f40a9ea0 0x7f90f40af640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:51164/0 (socket says 172.21.15.103:51164) 2024-08-19T13:15:01.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f90fa80b640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f90f40aa280 0x7f90f4143170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f90fa00a640 1 -- 172.21.15.103:0/2431242017 learned_addr learned my addr 172.21.15.103:0/2431242017 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:15:01.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f1427fff640 1 -- 172.21.15.103:0/2778699009 shutdown_connections 2024-08-19T13:15:01.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f1427fff640 1 -- 172.21.15.103:0/2778699009 wait complete. 2024-08-19T13:15:01.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7fd21d05c640 1 -- 172.21.15.103:0/2487406249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2100c4800 msgr2=0x7fd2100c4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7fc4cad7a640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4c4149f90 con 0x7fc4c40b16e0 2024-08-19T13:15:01.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7fc4cad7a640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c40b46e0 con 0x7fc4c40b4300 2024-08-19T13:15:01.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7fc4cad7a640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c40aa2e0 con 0x7fc4c40b2320 2024-08-19T13:15:01.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7fc4c3fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4c40b2320 0x7fc4c40adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7fc4c3fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4c40b2320 0x7fc4c40adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48816/0 (socket says 172.21.15.103:48816) 2024-08-19T13:15:01.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7fc4c3fff640 1 -- 172.21.15.103:0/4225450703 learned_addr learned my addr 172.21.15.103:0/4225450703 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7fc4c8b16640 1 --2- 172.21.15.103:0/4225450703 >> v2:172.21.15.103:3300/0 conn(0x7fc4c40b4300 0x7fc4c40b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.225+0000 7fc4c3fff640 1 -- 172.21.15.103:0/4225450703 >> v2:172.21.15.103:3300/0 conn(0x7fc4c40b4300 msgr2=0x7fc4c40b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:15:01.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7fc4c3fff640 1 --2- 172.21.15.103:0/4225450703 >> v2:172.21.15.103:3300/0 conn(0x7fc4c40b4300 0x7fc4c40b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7fc4c3fff640 1 -- 172.21.15.103:0/4225450703 >> v1:172.21.15.103:6789/0 conn(0x7fc4c40b16e0 legacy=0x7fc4c40af5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T13:15:01.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7fc4c3fff640 1 -- 172.21.15.103:0/4225450703 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c4073d00 con 0x7fc4c40b2320 2024-08-19T13:15:01.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7fc4c8b16640 1 --2- 172.21.15.103:0/4225450703 >> v2:172.21.15.103:3300/0 conn(0x7fc4c40b4300 0x7fc4c40b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:15:01.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7fd21d05c640 1 --2- 172.21.15.103:0/2487406249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2100c4800 0x7fd2100c4c70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd210006eb0 tx=0x7fd20c02a200 comp rx=0 tx=0).stop 2024-08-19T13:15:01.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.227+0000 7ff58124e640 1 Processor -- start 2024-08-19T13:15:01.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.227+0000 7f90fa80b640 1 -- 172.21.15.103:0/2431242017 >> v1:172.21.15.122:6789/0 conn(0x7f90f40ab090 legacy=0x7f90f40b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:15:01.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.227+0000 7f90fa80b640 1 -- 172.21.15.103:0/2431242017 >> v1:172.21.15.103:6789/0 conn(0x7f90f40a9ea0 legacy=0x7f90f40af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.227+0000 7fd21d05c640 1 -- 172.21.15.103:0/2487406249 shutdown_connections 2024-08-19T13:15:01.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.227+0000 7fd21d05c640 1 --2- 172.21.15.103:0/2487406249 >> v2:172.21.15.122:3300/0 conn(0x7fd2100caa10 0x7fd2100cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.227+0000 7fd21d05c640 1 --2- 172.21.15.103:0/2487406249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2100c4800 0x7fd2100c4c70 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd210006eb0 tx=0x7fd20c02a200 comp rx=0 tx=0).stop 2024-08-19T13:15:01.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.227+0000 7fd21d05c640 1 -- 172.21.15.103:0/2487406249 >> 172.21.15.103:0/2487406249 conn(0x7fd210005d30 msgr2=0x7fd210006140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.227+0000 7f33b14ae640 1 Processor -- start 2024-08-19T13:15:01.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.227+0000 7f90fa80b640 1 -- 172.21.15.103:0/2431242017 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f40b0ba0 con 0x7f90f40aa280 2024-08-19T13:15:01.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.227+0000 7fd21d05c640 1 -- 172.21.15.103:0/2487406249 shutdown_connections 2024-08-19T13:15:01.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.227+0000 7ff58124e640 1 -- start start 2024-08-19T13:15:01.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7ff58124e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c06c590 0x7ff57c0ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7f1427fff640 1 Processor -- start 2024-08-19T13:15:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7ff58124e640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff57c070440 0x7ff57c11ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f2f1acb5640 1 -- 172.21.15.103:0/3464848685 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f1408af00 con 0x7f2f14070330 2024-08-19T13:15:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.227+0000 7f33b14ae640 1 -- start start 2024-08-19T13:15:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7f33b14ae640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f33ac072f00 0x7f33ac0ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7f33b14ae640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33ac13bbe0 0x7f33ac13c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7f33b14ae640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33ac0727e0 con 0x7f33ac13e940 2024-08-19T13:15:01.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7f33b14ae640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f33ac0732e0 con 0x7f33ac072f00 2024-08-19T13:15:01.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7f33b14ae640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f33ac13c540 con 0x7f33ac13bbe0 2024-08-19T13:15:01.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.227+0000 7fd21d05c640 1 -- 172.21.15.103:0/2487406249 wait complete. 2024-08-19T13:15:01.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7f90fa80b640 1 --2- 172.21.15.103:0/2431242017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f40aa280 0x7f90f4143170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009d90 tx=0x7f90ec02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8fce40961ce98c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7f1427fff640 1 -- start start 2024-08-19T13:15:01.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7ff57ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c06c590 0x7ff57c0ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7ff57ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c06c590 0x7ff57c0ac640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.103:48856/0 (socket says 172.21.15.103:48856) 2024-08-19T13:15:01.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7ff57ad9d640 1 -- 172.21.15.103:0/1722988566 learned_addr learned my addr 172.21.15.103:0/1722988566 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7ff57a59c640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff57c070440 0x7ff57c11ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7f33aa7fc640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33ac13bbe0 0x7f33ac13c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7f33aaffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f33ac072f00 0x7f33ac0ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7f33aa7fc640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33ac13bbe0 0x7f33ac13c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48830/0 (socket says 172.21.15.103:48830) 2024-08-19T13:15:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7f33aa7fc640 1 -- 172.21.15.103:0/2146331438 learned_addr learned my addr 172.21.15.103:0/2146331438 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7ff58124e640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff57c141dc0 0x7ff57c1d3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.226+0000 7f2f117fa640 1 -- 172.21.15.103:0/3464848685 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0401b070 con 0x7f2f14070330 2024-08-19T13:15:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f2f117fa640 1 -- 172.21.15.103:0/3464848685 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04012d80 con 0x7f2f14070330 2024-08-19T13:15:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f2f117fa640 1 -- 172.21.15.103:0/3464848685 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f04013040 con 0x7f2f14070330 2024-08-19T13:15:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7fd21d05c640 1 Processor -- start 2024-08-19T13:15:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f1427fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14200360d0 0x7f142002f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7ff57b59e640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff57c141dc0 0x7ff57c1d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f33aa7fc640 1 -- 172.21.15.103:0/2146331438 >> v2:172.21.15.122:3300/0 conn(0x7f33ac072f00 msgr2=0x7f33ac0ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:15:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f33aa7fc640 1 --2- 172.21.15.103:0/2146331438 >> v2:172.21.15.122:3300/0 conn(0x7f33ac072f00 0x7f33ac0ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7ff58124e640 1 -- 172.21.15.103:0/1722988566 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c072280 con 0x7ff57c141dc0 2024-08-19T13:15:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7ff58124e640 1 -- 172.21.15.103:0/1722988566 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c06f1b0 con 0x7ff57c06c590 2024-08-19T13:15:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f33aa7fc640 1 -- 172.21.15.103:0/2146331438 >> v1:172.21.15.122:6789/0 conn(0x7f33ac13e940 legacy=0x7f33ac13edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7ff58124e640 1 -- 172.21.15.103:0/1722988566 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff57c072400 con 0x7ff57c070440 2024-08-19T13:15:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f1427fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1420037110 0x7f1420169b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f33aa7fc640 1 -- 172.21.15.103:0/2146331438 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac0aec60 con 0x7f33ac13bbe0 2024-08-19T13:15:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f1427fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14200cbef0 0x7f142016a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f1426ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14200360d0 0x7f142002f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f1426ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14200360d0 0x7f142002f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.103:48868/0 (socket says 172.21.15.103:48868) 2024-08-19T13:15:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f1426ffd640 1 -- 172.21.15.103:0/3579438612 learned_addr learned my addr 172.21.15.103:0/3579438612 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f2efaffd640 1 -- 172.21.15.103:0/3464848685 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f140ae7d0 con 0x7f2f14070330 2024-08-19T13:15:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f1427fff640 1 -- 172.21.15.103:0/3579438612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f14200c9290 con 0x7f14200cbef0 2024-08-19T13:15:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f2f117fa640 1 --2- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f2f0006bf30 0x7f2f0006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7f1427fff640 1 -- 172.21.15.103:0/3579438612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14200c9110 con 0x7f14200360d0 2024-08-19T13:15:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7f1427fff640 1 -- 172.21.15.103:0/3579438612 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f14200c9410 con 0x7f1420037110 2024-08-19T13:15:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7fd21d05c640 1 -- start start 2024-08-19T13:15:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7f2f18a51640 1 --2- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f2f0006bf30 0x7f2f0006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7f14267fc640 1 --2- 172.21.15.103:0/3579438612 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1420037110 0x7f1420169b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7f2f117fa640 1 -- 172.21.15.103:0/3464848685 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f2f0408adb0 con 0x7f2f14070330 2024-08-19T13:15:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7f90f9008640 1 -- 172.21.15.103:0/2431242017 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec037070 con 0x7f90f40aa280 2024-08-19T13:15:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7f90f9008640 1 -- 172.21.15.103:0/2431242017 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90ec032640 con 0x7f90f40aa280 2024-08-19T13:15:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7f90f9008640 1 -- 172.21.15.103:0/2431242017 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec032a10 con 0x7f90f40aa280 2024-08-19T13:15:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7f2f18a51640 1 --2- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f2f0006bf30 0x7f2f0006e3f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c001560 tx=0x7f2f0c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.228+0000 7fc4c3fff640 1 --2- 172.21.15.103:0/4225450703 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4c40b2320 0x7fc4c40adf70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009870 tx=0x7fc4b803b110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=230954cb67170aea server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7f14267fc640 1 -- 172.21.15.103:0/3579438612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14200360d0 msgr2=0x7f142002f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7f14267fc640 1 --2- 172.21.15.103:0/3579438612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14200360d0 0x7f142002f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7f14267fc640 1 -- 172.21.15.103:0/3579438612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14200cbef0 msgr2=0x7f142016a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7f14267fc640 1 --2- 172.21.15.103:0/3579438612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14200cbef0 0x7f142016a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7fc4c37fe640 1 -- 172.21.15.103:0/4225450703 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8048070 con 0x7fc4c40b2320 2024-08-19T13:15:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7fc4c37fe640 1 -- 172.21.15.103:0/4225450703 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b803be20 con 0x7fc4c40b2320 2024-08-19T13:15:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.229+0000 7fc4c37fe640 1 -- 172.21.15.103:0/4225450703 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8043660 con 0x7fc4c40b2320 2024-08-19T13:15:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7fc4cad7a640 1 -- 172.21.15.103:0/4225450703 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4c40b2320 msgr2=0x7fc4c40adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7fc4cad7a640 1 --2- 172.21.15.103:0/4225450703 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4c40b2320 0x7fc4c40adf70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009870 tx=0x7fc4b803b110 comp rx=0 tx=0).stop 2024-08-19T13:15:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7fc4cad7a640 1 -- 172.21.15.103:0/4225450703 shutdown_connections 2024-08-19T13:15:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7fc4cad7a640 1 --2- 172.21.15.103:0/4225450703 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4c40b2320 0x7fc4c40adf70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7fc4cad7a640 1 --2- 172.21.15.103:0/4225450703 >> v2:172.21.15.103:3300/0 conn(0x7fc4c40b4300 0x7fc4c40b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7fc4cad7a640 1 -- 172.21.15.103:0/4225450703 >> 172.21.15.103:0/4225450703 conn(0x7fc4c40b4ff0 msgr2=0x7fc4c40b5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7fc4cad7a640 1 -- 172.21.15.103:0/4225450703 shutdown_connections 2024-08-19T13:15:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.230+0000 7fc4cad7a640 1 -- 172.21.15.103:0/4225450703 wait complete. 2024-08-19T13:15:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7f33aa7fc640 1 --2- 172.21.15.103:0/2146331438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33ac13bbe0 0x7f33ac13c000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f339c00bb00 tx=0x7f339c02c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52692c2ea403ec12 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7f14267fc640 1 -- 172.21.15.103:0/3579438612 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14200300a0 con 0x7f1420037110 2024-08-19T13:15:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7f1426ffd640 1 --2- 172.21.15.103:0/3579438612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14200360d0 0x7f142002f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:15:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7ff57a59c640 1 -- 172.21.15.103:0/1722988566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c06c590 msgr2=0x7ff57c0ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7ff57a59c640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c06c590 0x7ff57c0ac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7ff57a59c640 1 -- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff57c141dc0 msgr2=0x7ff57c1d3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7ff57a59c640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff57c141dc0 0x7ff57c1d3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7ff57a59c640 1 -- 172.21.15.103:0/1722988566 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57c11f320 con 0x7ff57c070440 2024-08-19T13:15:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7f14267fc640 1 --2- 172.21.15.103:0/3579438612 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1420037110 0x7f1420169b00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f141c00e820 tx=0x7f141c00ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7fd21d05c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd210038b00 0x7fd21000c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7fd21d05c640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd2100caa10 0x7fd21000cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7ff57a59c640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff57c070440 0x7ff57c11ddb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff570034000 tx=0x7ff570004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7fd21d05c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd21000d2e0 0x7fd210009b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7fd21d05c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd2100c8390 con 0x7fd21000d2e0 2024-08-19T13:15:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7fd21d05c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd2100c8210 con 0x7fd210038b00 2024-08-19T13:15:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7fd21d05c640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fd2100d1da0 con 0x7fd2100caa10 2024-08-19T13:15:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7fd2177fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd2100caa10 0x7fd21000cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7fd2177fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd2100caa10 0x7fd21000cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48868/0 (socket says 172.21.15.103:48868) 2024-08-19T13:15:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7fd2177fe640 1 -- 172.21.15.103:0/4019024704 learned_addr learned my addr 172.21.15.103:0/4019024704 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7fd21c85b640 1 --2- 172.21.15.103:0/4019024704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd21000d2e0 0x7fd210009b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7f90fb00c640 1 -- 172.21.15.103:0/2431242017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f40aa280 msgr2=0x7f90f4143170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.231+0000 7f90fb00c640 1 --2- 172.21.15.103:0/2431242017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f40aa280 0x7f90f4143170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009d90 tx=0x7f90ec02a480 comp rx=0 tx=0).stop 2024-08-19T13:15:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.232+0000 7fd2177fe640 1 -- 172.21.15.103:0/4019024704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd210038b00 msgr2=0x7fd21000c6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T13:15:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.232+0000 7fd2177fe640 1 --2- 172.21.15.103:0/4019024704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd210038b00 0x7fd21000c6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.232+0000 7fc4cad7a640 1 Processor -- start 2024-08-19T13:15:01.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.232+0000 7fd2177fe640 1 -- 172.21.15.103:0/4019024704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd21000d2e0 msgr2=0x7fd210009b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.232+0000 7fd2177fe640 1 --2- 172.21.15.103:0/4019024704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd21000d2e0 0x7fd210009b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.232+0000 7f90fb00c640 1 -- 172.21.15.103:0/2431242017 shutdown_connections 2024-08-19T13:15:01.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.232+0000 7f90fb00c640 1 --2- 172.21.15.103:0/2431242017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f40aa280 0x7f90f4143170 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.232+0000 7f90fb00c640 1 -- 172.21.15.103:0/2431242017 >> 172.21.15.103:0/2431242017 conn(0x7f90f40b4c40 msgr2=0x7f90f40b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.232+0000 7fc4cad7a640 1 -- start start 2024-08-19T13:15:01.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.232+0000 7fd2177fe640 1 -- 172.21.15.103:0/4019024704 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2100cc1f0 con 0x7fd2100caa10 2024-08-19T13:15:01.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.232+0000 7f90fb00c640 1 -- 172.21.15.103:0/2431242017 shutdown_connections 2024-08-19T13:15:01.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.232+0000 7fd2177fe640 1 --2- 172.21.15.103:0/4019024704 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd2100caa10 0x7fd21000cda0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd20c02a7f0 tx=0x7fd20c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.232+0000 7f90fb00c640 1 -- 172.21.15.103:0/2431242017 wait complete. 2024-08-19T13:15:01.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4cad7a640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4c40b16e0 0x7fc4c4072ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4cad7a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4c40b4300 0x7fc4c41426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4cad7a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4c4142c10 0x7fc4c4070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4cad7a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c40aebd0 con 0x7fc4c40b4300 2024-08-19T13:15:01.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4cad7a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c40aea50 con 0x7fc4c4142c10 2024-08-19T13:15:01.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4cad7a640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c40ae4b0 con 0x7fc4c40b16e0 2024-08-19T13:15:01.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4c3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4c40b4300 0x7fc4c41426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4c8b16640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4c40b16e0 0x7fc4c4072ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4c3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4c40b4300 0x7fc4c41426d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50026/0 (socket says 172.21.15.103:50026) 2024-08-19T13:15:01.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4c3fff640 1 -- 172.21.15.103:0/1703134621 learned_addr learned my addr 172.21.15.103:0/1703134621 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4c9317640 1 --2- 172.21.15.103:0/1703134621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4c4142c10 0x7fc4c4070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4c3fff640 1 -- 172.21.15.103:0/1703134621 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4c40b16e0 msgr2=0x7fc4c4072ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4c3fff640 1 --2- 172.21.15.103:0/1703134621 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4c40b16e0 0x7fc4c4072ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4c3fff640 1 -- 172.21.15.103:0/1703134621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4c4142c10 msgr2=0x7fc4c4070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4c3fff640 1 --2- 172.21.15.103:0/1703134621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4c4142c10 0x7fc4c4070760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4c3fff640 1 -- 172.21.15.103:0/1703134621 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c4071e90 con 0x7fc4c40b4300 2024-08-19T13:15:01.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4c3fff640 1 --2- 172.21.15.103:0/1703134621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4c40b4300 0x7fc4c41426d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009870 tx=0x7fc4b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7f90fb00c640 1 Processor -- start 2024-08-19T13:15:01.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.233+0000 7fc4c17fa640 1 -- 172.21.15.103:0/1703134621 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8052070 con 0x7fc4c40b4300 2024-08-19T13:15:01.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.234+0000 7fc4cad7a640 1 -- 172.21.15.103:0/1703134621 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c4072170 con 0x7fc4c40b4300 2024-08-19T13:15:01.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.234+0000 7fc4c17fa640 1 -- 172.21.15.103:0/1703134621 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b804d460 con 0x7fc4c40b4300 2024-08-19T13:15:01.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.234+0000 7fc4c17fa640 1 -- 172.21.15.103:0/1703134621 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8048040 con 0x7fc4c40b4300 2024-08-19T13:15:01.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.234+0000 7fc4cad7a640 1 -- 172.21.15.103:0/1703134621 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c4072730 con 0x7fc4c40b4300 2024-08-19T13:15:01.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.234+0000 7f90fb00c640 1 -- start start 2024-08-19T13:15:01.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.234+0000 7f90fb00c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f40a9ea0 0x7f90f41cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90fa00a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f40a9ea0 0x7f90f41cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90fa00a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f40a9ea0 0x7f90f41cffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50042/0 (socket says 172.21.15.103:50042) 2024-08-19T13:15:01.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7fc4cad7a640 1 -- 172.21.15.103:0/1703134621 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c40adf70 con 0x7fc4c40b4300 2024-08-19T13:15:01.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90fa00a640 1 -- 172.21.15.103:0/23435365 learned_addr learned my addr 172.21.15.103:0/23435365 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90fb00c640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90f40aa280 0x7f90f41dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90fb00c640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90f40ab090 0x7f90f41e04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90fb00c640 1 -- 172.21.15.103:0/23435365 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90f414ad40 con 0x7f90f40a9ea0 2024-08-19T13:15:01.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90fb00c640 1 -- 172.21.15.103:0/23435365 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f90f414abc0 con 0x7f90f40ab090 2024-08-19T13:15:01.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90fb00c640 1 -- 172.21.15.103:0/23435365 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f90f414aec0 con 0x7f90f40aa280 2024-08-19T13:15:01.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90f9809640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90f40aa280 0x7f90f41dffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90fa80b640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90f40ab090 0x7f90f41e04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90fa00a640 1 -- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90f40aa280 msgr2=0x7f90f41dffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90fa00a640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90f40aa280 0x7f90f41dffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90fa00a640 1 -- 172.21.15.103:0/23435365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90f40ab090 msgr2=0x7f90f41e04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90fa00a640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90f40ab090 0x7f90f41e04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.235+0000 7f90fa00a640 1 -- 172.21.15.103:0/23435365 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f41d2050 con 0x7f90f40a9ea0 2024-08-19T13:15:01.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.236+0000 7fc4c17fa640 1 -- 172.21.15.103:0/1703134621 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4b8055070 con 0x7fc4c40b4300 2024-08-19T13:15:01.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.236+0000 7f90fa00a640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f40a9ea0 0x7f90f41cffc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f90f000c780 tx=0x7f90f000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.236+0000 7f90eaffd640 1 -- 172.21.15.103:0/23435365 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0018070 con 0x7f90f40a9ea0 2024-08-19T13:15:01.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.236+0000 7f90fb00c640 1 -- 172.21.15.103:0/23435365 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f413fe30 con 0x7f90f40a9ea0 2024-08-19T13:15:01.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.236+0000 7f90fb00c640 1 -- 172.21.15.103:0/23435365 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f41d2650 con 0x7f90f40a9ea0 2024-08-19T13:15:01.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.236+0000 7fc4c17fa640 1 --2- 172.21.15.103:0/1703134621 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc4b006be30 0x7fc4b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.236+0000 7fc4c8b16640 1 --2- 172.21.15.103:0/1703134621 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc4b006be30 0x7fc4b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.236+0000 7fc4c17fa640 1 -- 172.21.15.103:0/1703134621 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7fc4b80bbf60 con 0x7fc4c40b4300 2024-08-19T13:15:01.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.237+0000 7fc4c8b16640 1 --2- 172.21.15.103:0/1703134621 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc4b006be30 0x7fc4b006e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009bd0 tx=0x7fc4b400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.237+0000 7f90fb00c640 1 -- 172.21.15.103:0/23435365 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f40abb30 con 0x7f90f40a9ea0 2024-08-19T13:15:01.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.238+0000 7f33a9ffb640 1 -- 172.21.15.103:0/2146331438 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c03a070 con 0x7f33ac13bbe0 2024-08-19T13:15:01.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.238+0000 7ff5637fe640 1 -- 172.21.15.103:0/1722988566 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570045070 con 0x7ff57c070440 2024-08-19T13:15:01.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.238+0000 7ff58124e640 1 -- 172.21.15.103:0/1722988566 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57c11d890 con 0x7ff57c070440 2024-08-19T13:15:01.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.238+0000 7f2f117fa640 1 -- 172.21.15.103:0/3464848685 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f2f04059330 con 0x7f2f14070330 2024-08-19T13:15:01.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.238+0000 7ff58124e640 1 -- 172.21.15.103:0/1722988566 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff57c11d4a0 con 0x7ff57c070440 2024-08-19T13:15:01.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.238+0000 7fd214ff9640 1 -- 172.21.15.103:0/4019024704 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c037040 con 0x7fd2100caa10 2024-08-19T13:15:01.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.238+0000 7fd21d05c640 1 -- 172.21.15.103:0/4019024704 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2100cc460 con 0x7fd2100caa10 2024-08-19T13:15:01.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.238+0000 7fd21d05c640 1 -- 172.21.15.103:0/4019024704 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2100cca20 con 0x7fd2100caa10 2024-08-19T13:15:01.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.238+0000 7ff5637fe640 1 -- 172.21.15.103:0/1722988566 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff570032620 con 0x7ff57c070440 2024-08-19T13:15:01.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.239+0000 7ff5637fe640 1 -- 172.21.15.103:0/1722988566 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570032070 con 0x7ff57c070440 2024-08-19T13:15:01.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.238+0000 7f33a9ffb640 1 -- 172.21.15.103:0/2146331438 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339c032480 con 0x7f33ac13bbe0 2024-08-19T13:15:01.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.239+0000 7f33b14ae640 1 -- 172.21.15.103:0/2146331438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33ac13bbe0 msgr2=0x7f33ac13c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.239+0000 7f33b14ae640 1 --2- 172.21.15.103:0/2146331438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33ac13bbe0 0x7f33ac13c000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f339c00bb00 tx=0x7f339c02c6e0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.238+0000 7f140ffff640 1 -- 172.21.15.103:0/3579438612 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c017070 con 0x7f1420037110 2024-08-19T13:15:01.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.238+0000 7f1427fff640 1 -- 172.21.15.103:0/3579438612 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142016a6a0 con 0x7f1420037110 2024-08-19T13:15:01.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.238+0000 7f1427fff640 1 -- 172.21.15.103:0/3579438612 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f142002fb90 con 0x7f1420037110 2024-08-19T13:15:01.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.239+0000 7f140ffff640 1 -- 172.21.15.103:0/3579438612 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f141c00c8e0 con 0x7f1420037110 2024-08-19T13:15:01.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.240+0000 7f140ffff640 1 -- 172.21.15.103:0/3579438612 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c017070 con 0x7f1420037110 2024-08-19T13:15:01.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.240+0000 7f1427fff640 1 -- 172.21.15.103:0/3579438612 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1408005080 con 0x7f1420037110 2024-08-19T13:15:01.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.240+0000 7fd214ff9640 1 -- 172.21.15.103:0/4019024704 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20c032440 con 0x7fd2100caa10 2024-08-19T13:15:01.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.240+0000 7fd214ff9640 1 -- 172.21.15.103:0/4019024704 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c004450 con 0x7fd2100caa10 2024-08-19T13:15:01.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.241+0000 7ff58124e640 1 -- 172.21.15.103:0/1722988566 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff55c005080 con 0x7ff57c070440 2024-08-19T13:15:01.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.241+0000 7ff5637fe640 1 -- 172.21.15.103:0/1722988566 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff570048070 con 0x7ff57c070440 2024-08-19T13:15:01.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.241+0000 7ff5637fe640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff56806be60 0x7ff56806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.241+0000 7ff5637fe640 1 -- 172.21.15.103:0/1722988566 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7ff5700aad20 con 0x7ff57c070440 2024-08-19T13:15:01.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.241+0000 7fd21d05c640 1 -- 172.21.15.103:0/4019024704 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f8005080 con 0x7fd2100caa10 2024-08-19T13:15:01.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.242+0000 7ff57ad9d640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff56806be60 0x7ff56806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.242+0000 7f33b14ae640 1 -- 172.21.15.103:0/2146331438 shutdown_connections 2024-08-19T13:15:01.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.242+0000 7f33b14ae640 1 --2- 172.21.15.103:0/2146331438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33ac13bbe0 0x7f33ac13c000 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.242+0000 7f33b14ae640 1 --2- 172.21.15.103:0/2146331438 >> v2:172.21.15.122:3300/0 conn(0x7f33ac072f00 0x7f33ac0ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.242+0000 7f33b14ae640 1 -- 172.21.15.103:0/2146331438 >> 172.21.15.103:0/2146331438 conn(0x7f33ac0b4a20 msgr2=0x7f33ac0b4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.242+0000 7f33b14ae640 1 -- 172.21.15.103:0/2146331438 shutdown_connections 2024-08-19T13:15:01.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.242+0000 7f33b14ae640 1 -- 172.21.15.103:0/2146331438 wait complete. 2024-08-19T13:15:01.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.242+0000 7ff57ad9d640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff56806be60 0x7ff56806e320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009c00 tx=0x7ff56c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.242+0000 7fd214ff9640 1 -- 172.21.15.103:0/4019024704 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd20c003610 con 0x7fd2100caa10 2024-08-19T13:15:01.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.243+0000 7fd214ff9640 1 --2- 172.21.15.103:0/4019024704 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fd1fc06be30 0x7fd1fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.243+0000 7fd214ff9640 1 -- 172.21.15.103:0/4019024704 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7fd20c0ab880 con 0x7fd2100caa10 2024-08-19T13:15:01.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.243+0000 7fd217fff640 1 --2- 172.21.15.103:0/4019024704 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fd1fc06be30 0x7fd1fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.244+0000 7f90eaffd640 1 -- 172.21.15.103:0/23435365 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f00072d0 con 0x7f90f40a9ea0 2024-08-19T13:15:01.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.244+0000 7f90eaffd640 1 -- 172.21.15.103:0/23435365 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0022060 con 0x7f90f40a9ea0 2024-08-19T13:15:01.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.244+0000 7f33b14ae640 1 Processor -- start 2024-08-19T13:15:01.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.244+0000 7fd217fff640 1 --2- 172.21.15.103:0/4019024704 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fd1fc06be30 0x7fd1fc06e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd204000c00 tx=0x7fd204007890 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.244+0000 7f33b14ae640 1 -- start start 2024-08-19T13:15:01.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.244+0000 7fc4c17fa640 1 -- 172.21.15.103:0/1703134621 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fc4b808a480 con 0x7fc4c40b4300 2024-08-19T13:15:01.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.245+0000 7f33b14ae640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33ac072f00 0x7f33ac140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.245+0000 7f33b14ae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33ac13bbe0 0x7f33ac1d34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.245+0000 7f33b14ae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33ac13e940 0x7f33ac1d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.245+0000 7f33b14ae640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac073e70 con 0x7f33ac13e940 2024-08-19T13:15:01.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.245+0000 7f33b14ae640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac073ab0 con 0x7f33ac13bbe0 2024-08-19T13:15:01.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.245+0000 7f33b14ae640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f33ac0676f0 con 0x7f33ac072f00 2024-08-19T13:15:01.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.245+0000 7f33ab7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33ac13e940 0x7f33ac1d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.245+0000 7f33ab7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33ac13e940 0x7f33ac1d3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50048/0 (socket says 172.21.15.103:50048) 2024-08-19T13:15:01.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.245+0000 7f33aaffd640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33ac072f00 0x7f33ac140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.245+0000 7f33ab7fe640 1 -- 172.21.15.103:0/362590752 learned_addr learned my addr 172.21.15.103:0/362590752 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.246+0000 7f33aa7fc640 1 --2- 172.21.15.103:0/362590752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33ac13bbe0 0x7f33ac1d34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.246+0000 7f33aaffd640 1 -- 172.21.15.103:0/362590752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33ac13bbe0 msgr2=0x7f33ac1d34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.246+0000 7f33aaffd640 1 --2- 172.21.15.103:0/362590752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33ac13bbe0 0x7f33ac1d34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.246+0000 7f33aaffd640 1 -- 172.21.15.103:0/362590752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33ac13e940 msgr2=0x7f33ac1d3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.246+0000 7f33aaffd640 1 --2- 172.21.15.103:0/362590752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33ac13e940 0x7f33ac1d3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.246+0000 7f33aaffd640 1 -- 172.21.15.103:0/362590752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac141f60 con 0x7f33ac072f00 2024-08-19T13:15:01.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.246+0000 7f33aaffd640 1 --2- 172.21.15.103:0/362590752 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33ac072f00 0x7f33ac140910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f33a4007e90 tx=0x7f33a400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.247+0000 7f90eaffd640 1 -- 172.21.15.103:0/23435365 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90f001b070 con 0x7f90f40a9ea0 2024-08-19T13:15:01.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.247+0000 7f90eaffd640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f90d806be30 0x7f90d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.247+0000 7f90eaffd640 1 -- 172.21.15.103:0/23435365 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f90f008a710 con 0x7f90f40a9ea0 2024-08-19T13:15:01.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.247+0000 7f90f9809640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f90d806be30 0x7f90d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.248+0000 7ff5637fe640 1 -- 172.21.15.103:0/1722988566 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7ff570079190 con 0x7ff57c070440 2024-08-19T13:15:01.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.248+0000 7f90f9809640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f90d806be30 0x7f90d806e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f90f413eed0 tx=0x7f90e4009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.249+0000 7f140ffff640 1 -- 172.21.15.103:0/3579438612 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f141c01a070 con 0x7f1420037110 2024-08-19T13:15:01.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.249+0000 7f140ffff640 1 --2- 172.21.15.103:0/3579438612 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f141406c030 0x7f141406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.250+0000 7f1426ffd640 1 --2- 172.21.15.103:0/3579438612 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f141406c030 0x7f141406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.250+0000 7f1426ffd640 1 --2- 172.21.15.103:0/3579438612 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f141406c030 0x7f141406e4f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1428000b60 tx=0x7f142806a980 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.253+0000 7f90eaffd640 1 -- 172.21.15.103:0/23435365 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f90f0058c90 con 0x7f90f40a9ea0 2024-08-19T13:15:01.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.257+0000 7fd214ff9640 1 -- 172.21.15.103:0/4019024704 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fd210030740 con 0x7fd2100caa10 2024-08-19T13:15:01.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.257+0000 7f140ffff640 1 -- 172.21.15.103:0/3579438612 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f141c057420 con 0x7f1420037110 2024-08-19T13:15:01.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.256+0000 7f33937fe640 1 -- 172.21.15.103:0/362590752 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4013760 con 0x7f33ac072f00 2024-08-19T13:15:01.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.257+0000 7f33b14ae640 1 -- 172.21.15.103:0/362590752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ac142240 con 0x7f33ac072f00 2024-08-19T13:15:01.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.257+0000 7f33b14ae640 1 -- 172.21.15.103:0/362590752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ac142940 con 0x7f33ac072f00 2024-08-19T13:15:01.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.258+0000 7f33937fe640 1 -- 172.21.15.103:0/362590752 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a400ee70 con 0x7f33ac072f00 2024-08-19T13:15:01.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.258+0000 7f33937fe640 1 -- 172.21.15.103:0/362590752 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4029060 con 0x7f33ac072f00 2024-08-19T13:15:01.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.261+0000 7f33917fa640 1 -- 172.21.15.103:0/362590752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33ac067b20 con 0x7f33ac072f00 2024-08-19T13:15:01.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.263+0000 7f140ffff640 1 -- 172.21.15.103:0/3579438612 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f141c059380 con 0x7f1420037110 2024-08-19T13:15:01.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.264+0000 7f33937fe640 1 -- 172.21.15.103:0/362590752 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33a4013b00 con 0x7f33ac072f00 2024-08-19T13:15:01.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.265+0000 7f33937fe640 1 --2- 172.21.15.103:0/362590752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f339806be30 0x7f339806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.265+0000 7f33937fe640 1 -- 172.21.15.103:0/362590752 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f33a408fdd0 con 0x7f33ac072f00 2024-08-19T13:15:01.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.266+0000 7f33aa7fc640 1 --2- 172.21.15.103:0/362590752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f339806be30 0x7f339806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.267+0000 7f33aa7fc640 1 --2- 172.21.15.103:0/362590752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f339806be30 0x7f339806e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f339c037000 tx=0x7f339c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.272+0000 7f33937fe640 1 -- 172.21.15.103:0/362590752 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f33a405e350 con 0x7f33ac072f00 2024-08-19T13:15:01.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.566+0000 7f3f5474b640 1 -- 172.21.15.103:0/3898367543 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3f4c075c30 con 0x7f3f4c070330 2024-08-19T13:15:01.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.567+0000 7f3f42ffd640 1 -- 172.21.15.103:0/3898367543 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3f3c059040 con 0x7f3f4c070330 2024-08-19T13:15:01.570 INFO:teuthology.orchestra.run.smithi103.stdout:55834574852 2024-08-19T13:15:01.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.570+0000 7f3f5474b640 1 -- 172.21.15.103:0/3898367543 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f3f3006c0a0 msgr2=0x7f3f3006e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.570+0000 7f3f5474b640 1 --2- 172.21.15.103:0/3898367543 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f3f3006c0a0 0x7f3f3006e560 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3f48000f60 tx=0x7f3f48000bc0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.570+0000 7f3f5474b640 1 -- 172.21.15.103:0/3898367543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f4c070330 msgr2=0x7f3f4c0ad290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.570+0000 7f3f5474b640 1 --2- 172.21.15.103:0/3898367543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f4c070330 0x7f3f4c0ad290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00c550 tx=0x7f3f3c00ca10 comp rx=0 tx=0).stop 2024-08-19T13:15:01.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.571+0000 7f3f5474b640 1 -- 172.21.15.103:0/3898367543 shutdown_connections 2024-08-19T13:15:01.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.571+0000 7f3f5474b640 1 --2- 172.21.15.103:0/3898367543 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f3f3006c0a0 0x7f3f3006e560 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.571+0000 7f3f5474b640 1 --2- 172.21.15.103:0/3898367543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f4c072560 0x7f3f4c0ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.571+0000 7f3f5474b640 1 --2- 172.21.15.103:0/3898367543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f4c070330 0x7f3f4c0ad290 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.571+0000 7f3f5474b640 1 --2- 172.21.15.103:0/3898367543 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f4c06f500 0x7f3f4c0ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.571+0000 7f3f5474b640 1 -- 172.21.15.103:0/3898367543 >> 172.21.15.103:0/3898367543 conn(0x7f3f4c0b3fb0 msgr2=0x7f3f4c0b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.571+0000 7f3f5474b640 1 -- 172.21.15.103:0/3898367543 shutdown_connections 2024-08-19T13:15:01.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.571+0000 7f3f5474b640 1 -- 172.21.15.103:0/3898367543 wait complete. 2024-08-19T13:15:01.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2024-08-19T13:15:01.587 DEBUG:teuthology.parallel:result is None 2024-08-19T13:15:01.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.608+0000 7fc4cad7a640 1 -- 172.21.15.103:0/1703134621 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc4c4075980 con 0x7fc4c40b4300 2024-08-19T13:15:01.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.608+0000 7fc4c17fa640 1 -- 172.21.15.103:0/1703134621 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc4b803be20 con 0x7fc4c40b4300 2024-08-19T13:15:01.609 INFO:teuthology.orchestra.run.smithi103.stdout:55834574851 2024-08-19T13:15:01.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.610+0000 7fc4cad7a640 1 -- 172.21.15.103:0/1703134621 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc4b006be30 msgr2=0x7fc4b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.610+0000 7fc4cad7a640 1 --2- 172.21.15.103:0/1703134621 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc4b006be30 0x7fc4b006e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009bd0 tx=0x7fc4b400b040 comp rx=0 tx=0).stop 2024-08-19T13:15:01.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.611+0000 7fc4cad7a640 1 -- 172.21.15.103:0/1703134621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4c40b4300 msgr2=0x7fc4c41426d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.611+0000 7fc4cad7a640 1 --2- 172.21.15.103:0/1703134621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4c40b4300 0x7fc4c41426d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009870 tx=0x7fc4b80046c0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.611+0000 7fc4cad7a640 1 -- 172.21.15.103:0/1703134621 shutdown_connections 2024-08-19T13:15:01.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.611+0000 7fc4cad7a640 1 --2- 172.21.15.103:0/1703134621 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc4b006be30 0x7fc4b006e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.611+0000 7fc4cad7a640 1 --2- 172.21.15.103:0/1703134621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4c4142c10 0x7fc4c4070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.611+0000 7fc4cad7a640 1 --2- 172.21.15.103:0/1703134621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4c40b4300 0x7fc4c41426d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.611+0000 7fc4cad7a640 1 --2- 172.21.15.103:0/1703134621 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4c40b16e0 0x7fc4c4072ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.611+0000 7fc4cad7a640 1 -- 172.21.15.103:0/1703134621 >> 172.21.15.103:0/1703134621 conn(0x7fc4c40b4ff0 msgr2=0x7fc4c4148250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.611+0000 7fc4cad7a640 1 -- 172.21.15.103:0/1703134621 shutdown_connections 2024-08-19T13:15:01.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.611+0000 7fc4cad7a640 1 -- 172.21.15.103:0/1703134621 wait complete. 2024-08-19T13:15:01.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2024-08-19T13:15:01.623 DEBUG:teuthology.parallel:result is None 2024-08-19T13:15:01.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.636+0000 7f2efaffd640 1 -- 172.21.15.103:0/3464848685 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2f140008d0 con 0x7f2f14070330 2024-08-19T13:15:01.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.637+0000 7f2f117fa640 1 -- 172.21.15.103:0/3464848685 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2f0405d630 con 0x7f2f14070330 2024-08-19T13:15:01.637 INFO:teuthology.orchestra.run.smithi103.stdout:55834574852 2024-08-19T13:15:01.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.640+0000 7f2efaffd640 1 -- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f2f0006bf30 msgr2=0x7f2f0006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.640+0000 7f2efaffd640 1 --2- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f2f0006bf30 0x7f2f0006e3f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c001560 tx=0x7f2f0c009040 comp rx=0 tx=0).stop 2024-08-19T13:15:01.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.640+0000 7f2efaffd640 1 -- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2f14070330 msgr2=0x7f2f1408a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.640+0000 7f2efaffd640 1 --2- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2f14070330 0x7f2f1408a480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2f040149a0 tx=0x7f2f04014e60 comp rx=0 tx=0).stop 2024-08-19T13:15:01.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.640+0000 7f2efaffd640 1 -- 172.21.15.103:0/3464848685 shutdown_connections 2024-08-19T13:15:01.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.640+0000 7f2efaffd640 1 --2- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f2f0006bf30 0x7f2f0006e3f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.640+0000 7f2efaffd640 1 --2- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1413d9c0 0x7f2f1408a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.640+0000 7f2efaffd640 1 --2- 172.21.15.103:0/3464848685 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2f14070330 0x7f2f1408a480 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.640+0000 7f2efaffd640 1 --2- 172.21.15.103:0/3464848685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f1406f290 0x7f2f140adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.640+0000 7f2efaffd640 1 -- 172.21.15.103:0/3464848685 >> 172.21.15.103:0/3464848685 conn(0x7f2f140b3fb0 msgr2=0x7f2f1414d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.640+0000 7f2efaffd640 1 -- 172.21.15.103:0/3464848685 shutdown_connections 2024-08-19T13:15:01.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.640+0000 7f2efaffd640 1 -- 172.21.15.103:0/3464848685 wait complete. 2024-08-19T13:15:01.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.653+0000 7ff58124e640 1 -- 172.21.15.103:0/1722988566 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff55c005370 con 0x7ff57c070440 2024-08-19T13:15:01.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.653+0000 7ff5637fe640 1 -- 172.21.15.103:0/1722988566 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff57007d490 con 0x7ff57c070440 2024-08-19T13:15:01.654 INFO:teuthology.orchestra.run.smithi103.stdout:55834574853 2024-08-19T13:15:01.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2024-08-19T13:15:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.655+0000 7f90fb00c640 1 -- 172.21.15.103:0/23435365 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f90f4143150 con 0x7f90f40a9ea0 2024-08-19T13:15:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.655+0000 7f90eaffd640 1 -- 172.21.15.103:0/23435365 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f90f005cf90 con 0x7f90f40a9ea0 2024-08-19T13:15:01.656 DEBUG:teuthology.parallel:result is None 2024-08-19T13:15:01.656 INFO:teuthology.orchestra.run.smithi103.stdout:55834574852 2024-08-19T13:15:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.656+0000 7ff58124e640 1 -- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff56806be60 msgr2=0x7ff56806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.656+0000 7ff58124e640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff56806be60 0x7ff56806e320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009c00 tx=0x7ff56c009190 comp rx=0 tx=0).stop 2024-08-19T13:15:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.656+0000 7ff58124e640 1 -- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff57c070440 msgr2=0x7ff57c11ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.656+0000 7ff58124e640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff57c070440 0x7ff57c11ddb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff570034000 tx=0x7ff570004730 comp rx=0 tx=0).stop 2024-08-19T13:15:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.656+0000 7ff58124e640 1 -- 172.21.15.103:0/1722988566 shutdown_connections 2024-08-19T13:15:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.656+0000 7ff58124e640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff56806be60 0x7ff56806e320 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.656+0000 7ff58124e640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff57c141dc0 0x7ff57c1d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.656+0000 7ff58124e640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff57c070440 0x7ff57c11ddb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.656+0000 7ff58124e640 1 --2- 172.21.15.103:0/1722988566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c06c590 0x7ff57c0ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7ff58124e640 1 -- 172.21.15.103:0/1722988566 >> 172.21.15.103:0/1722988566 conn(0x7ff57c06b800 msgr2=0x7ff57c06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7ff58124e640 1 -- 172.21.15.103:0/1722988566 shutdown_connections 2024-08-19T13:15:01.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7ff58124e640 1 -- 172.21.15.103:0/1722988566 wait complete. 2024-08-19T13:15:01.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7f90fb00c640 1 -- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f90d806be30 msgr2=0x7f90d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7f90fb00c640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f90d806be30 0x7f90d806e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f90f413eed0 tx=0x7f90e4009060 comp rx=0 tx=0).stop 2024-08-19T13:15:01.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7f90fb00c640 1 -- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f40a9ea0 msgr2=0x7f90f41cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7f90fb00c640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f40a9ea0 0x7f90f41cffc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f90f000c780 tx=0x7f90f000cc40 comp rx=0 tx=0).stop 2024-08-19T13:15:01.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7f90fb00c640 1 -- 172.21.15.103:0/23435365 shutdown_connections 2024-08-19T13:15:01.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7f90fb00c640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f90d806be30 0x7f90d806e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7f90fb00c640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90f40ab090 0x7f90f41e04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7f90fb00c640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90f40aa280 0x7f90f41dffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7f90fb00c640 1 --2- 172.21.15.103:0/23435365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f40a9ea0 0x7f90f41cffc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7f90fb00c640 1 -- 172.21.15.103:0/23435365 >> 172.21.15.103:0/23435365 conn(0x7f90f40b4c40 msgr2=0x7f90f4140770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7f90fb00c640 1 -- 172.21.15.103:0/23435365 shutdown_connections 2024-08-19T13:15:01.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.657+0000 7f90fb00c640 1 -- 172.21.15.103:0/23435365 wait complete. 2024-08-19T13:15:01.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.661+0000 7fd21d05c640 1 -- 172.21.15.103:0/4019024704 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd1f8005370 con 0x7fd2100caa10 2024-08-19T13:15:01.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.661+0000 7fd214ff9640 1 -- 172.21.15.103:0/4019024704 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd20c07a490 con 0x7fd2100caa10 2024-08-19T13:15:01.662 INFO:teuthology.orchestra.run.smithi103.stdout:55834574852 2024-08-19T13:15:01.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.663+0000 7f1427fff640 1 -- 172.21.15.103:0/3579438612 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1408005640 con 0x7f1420037110 2024-08-19T13:15:01.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.664+0000 7f140ffff640 1 -- 172.21.15.103:0/3579438612 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f141c05d680 con 0x7f1420037110 2024-08-19T13:15:01.664 INFO:teuthology.orchestra.run.smithi103.stdout:55834574852 2024-08-19T13:15:01.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.665+0000 7fd21d05c640 1 -- 172.21.15.103:0/4019024704 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fd1fc06be30 msgr2=0x7fd1fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.665+0000 7fd21d05c640 1 --2- 172.21.15.103:0/4019024704 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fd1fc06be30 0x7fd1fc06e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd204000c00 tx=0x7fd204007890 comp rx=0 tx=0).stop 2024-08-19T13:15:01.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.665+0000 7fd21d05c640 1 -- 172.21.15.103:0/4019024704 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd2100caa10 msgr2=0x7fd21000cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.665+0000 7fd21d05c640 1 --2- 172.21.15.103:0/4019024704 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd2100caa10 0x7fd21000cda0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd20c02a7f0 tx=0x7fd20c0046c0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.665+0000 7fd21d05c640 1 -- 172.21.15.103:0/4019024704 shutdown_connections 2024-08-19T13:15:01.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.665+0000 7fd21d05c640 1 --2- 172.21.15.103:0/4019024704 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fd1fc06be30 0x7fd1fc06e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.665+0000 7fd21d05c640 1 --2- 172.21.15.103:0/4019024704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd21000d2e0 0x7fd210009b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.665+0000 7fd21d05c640 1 --2- 172.21.15.103:0/4019024704 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd2100caa10 0x7fd21000cda0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.665+0000 7fd21d05c640 1 --2- 172.21.15.103:0/4019024704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd210038b00 0x7fd21000c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.665+0000 7fd21d05c640 1 -- 172.21.15.103:0/4019024704 >> 172.21.15.103:0/4019024704 conn(0x7fd210005d30 msgr2=0x7fd21002dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.665+0000 7fd21d05c640 1 -- 172.21.15.103:0/4019024704 shutdown_connections 2024-08-19T13:15:01.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.665+0000 7fd21d05c640 1 -- 172.21.15.103:0/4019024704 wait complete. 2024-08-19T13:15:01.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.666+0000 7f1427fff640 1 -- 172.21.15.103:0/3579438612 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f141406c030 msgr2=0x7f141406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.666+0000 7f1427fff640 1 --2- 172.21.15.103:0/3579438612 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f141406c030 0x7f141406e4f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1428000b60 tx=0x7f142806a980 comp rx=0 tx=0).stop 2024-08-19T13:15:01.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.666+0000 7f1427fff640 1 -- 172.21.15.103:0/3579438612 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1420037110 msgr2=0x7f1420169b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.666+0000 7f1427fff640 1 --2- 172.21.15.103:0/3579438612 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1420037110 0x7f1420169b00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f141c00e820 tx=0x7f141c00ece0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.667+0000 7f1427fff640 1 -- 172.21.15.103:0/3579438612 shutdown_connections 2024-08-19T13:15:01.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.667+0000 7f1427fff640 1 --2- 172.21.15.103:0/3579438612 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f141406c030 0x7f141406e4f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.667+0000 7f1427fff640 1 --2- 172.21.15.103:0/3579438612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14200cbef0 0x7f142016a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.667+0000 7f1427fff640 1 --2- 172.21.15.103:0/3579438612 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1420037110 0x7f1420169b00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.667+0000 7f1427fff640 1 --2- 172.21.15.103:0/3579438612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14200360d0 0x7f142002f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.667+0000 7f1427fff640 1 -- 172.21.15.103:0/3579438612 >> 172.21.15.103:0/3579438612 conn(0x7f142003a420 msgr2=0x7f14200377d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.667+0000 7f1427fff640 1 -- 172.21.15.103:0/3579438612 shutdown_connections 2024-08-19T13:15:01.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.667+0000 7f1427fff640 1 -- 172.21.15.103:0/3579438612 wait complete. 2024-08-19T13:15:01.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.669+0000 7f33917fa640 1 -- 172.21.15.103:0/362590752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f33ac142410 con 0x7f33ac072f00 2024-08-19T13:15:01.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.669+0000 7f33937fe640 1 -- 172.21.15.103:0/362590752 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f33a4062650 con 0x7f33ac072f00 2024-08-19T13:15:01.670 INFO:teuthology.orchestra.run.smithi103.stdout:55834574852 2024-08-19T13:15:01.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.3 2024-08-19T13:15:01.671 DEBUG:teuthology.parallel:result is None 2024-08-19T13:15:01.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-08-19T13:15:01.673 DEBUG:teuthology.parallel:result is None 2024-08-19T13:15:01.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.673+0000 7f33917fa640 1 -- 172.21.15.103:0/362590752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f339806be30 msgr2=0x7f339806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.673+0000 7f33917fa640 1 --2- 172.21.15.103:0/362590752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f339806be30 0x7f339806e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f339c037000 tx=0x7f339c030000 comp rx=0 tx=0).stop 2024-08-19T13:15:01.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.673+0000 7f33917fa640 1 -- 172.21.15.103:0/362590752 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33ac072f00 msgr2=0x7f33ac140910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.673+0000 7f33917fa640 1 --2- 172.21.15.103:0/362590752 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33ac072f00 0x7f33ac140910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f33a4007e90 tx=0x7f33a400e4b0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.673+0000 7f33917fa640 1 -- 172.21.15.103:0/362590752 shutdown_connections 2024-08-19T13:15:01.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.673+0000 7f33917fa640 1 --2- 172.21.15.103:0/362590752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f339806be30 0x7f339806e2f0 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f339c037000 tx=0x7f339c030000 comp rx=0 tx=0).stop 2024-08-19T13:15:01.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.673+0000 7f33917fa640 1 --2- 172.21.15.103:0/362590752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33ac13e940 0x7f33ac1d3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.673+0000 7f33917fa640 1 --2- 172.21.15.103:0/362590752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33ac13bbe0 0x7f33ac1d34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.673+0000 7f33917fa640 1 --2- 172.21.15.103:0/362590752 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f33ac072f00 0x7f33ac140910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.673+0000 7f33917fa640 1 -- 172.21.15.103:0/362590752 >> 172.21.15.103:0/362590752 conn(0x7f33ac0b4a20 msgr2=0x7f33ac137c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.673+0000 7f33917fa640 1 -- 172.21.15.103:0/362590752 shutdown_connections 2024-08-19T13:15:01.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.673+0000 7f33917fa640 1 -- 172.21.15.103:0/362590752 wait complete. 2024-08-19T13:15:01.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.6 2024-08-19T13:15:01.681 DEBUG:teuthology.parallel:result is None 2024-08-19T13:15:01.682 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2024-08-19T13:15:01.682 DEBUG:teuthology.parallel:result is None 2024-08-19T13:15:01.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2024-08-19T13:15:01.688 DEBUG:teuthology.parallel:result is None 2024-08-19T13:15:01.688 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-19T13:15:01.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T13:15:01.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.827+0000 7fe4fc9df640 1 Processor -- start 2024-08-19T13:15:01.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.827+0000 7fe4fc9df640 1 -- start start 2024-08-19T13:15:01.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.827+0000 7fe4fc9df640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f8071f10 0x7fe4f8072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.827+0000 7fe4fc9df640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe4f8140e80 0x7fe4f813d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.827+0000 7fe4fc9df640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4f8140960 con 0x7fe4f8071b30 2024-08-19T13:15:01.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.827+0000 7fe4fc9df640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fe4f80b1510 con 0x7fe4f8140e80 2024-08-19T13:15:01.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.827+0000 7fe4fc9df640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f8141310 con 0x7fe4f8071f10 2024-08-19T13:15:01.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.828+0000 7fe4f6d9d640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe4f8140e80 0x7fe4f813d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.828+0000 7fe4f5d9b640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f8071f10 0x7fe4f8072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.828+0000 7fe4f6d9d640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe4f8140e80 0x7fe4f813d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50058/0 (socket says 172.21.15.103:50058) 2024-08-19T13:15:01.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.828+0000 7fe4f5d9b640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f8071f10 0x7fe4f8072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48914/0 (socket says 172.21.15.103:48914) 2024-08-19T13:15:01.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.828+0000 7fe4f6d9d640 1 -- 172.21.15.103:0/3362706536 learned_addr learned my addr 172.21.15.103:0/3362706536 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.828+0000 7fe4f5d9b640 1 -- 172.21.15.103:0/3362706536 >> v2:172.21.15.103:3300/0 conn(0x7fe4f8140e80 msgr2=0x7fe4f813d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:15:01.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.828+0000 7fe4f5d9b640 1 --2- 172.21.15.103:0/3362706536 >> v2:172.21.15.103:3300/0 conn(0x7fe4f8140e80 0x7fe4f813d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.828+0000 7fe4f5d9b640 1 -- 172.21.15.103:0/3362706536 >> v1:172.21.15.103:6789/0 conn(0x7fe4f8071b30 legacy=0x7fe4f80b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.828+0000 7fe4f5d9b640 1 -- 172.21.15.103:0/3362706536 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f8070ee0 con 0x7fe4f8071f10 2024-08-19T13:15:01.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.828+0000 7fe4f6d9d640 1 --2- 172.21.15.103:0/3362706536 >> v2:172.21.15.103:3300/0 conn(0x7fe4f8140e80 0x7fe4f813d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:15:01.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.829+0000 7fe4f5d9b640 1 --2- 172.21.15.103:0/3362706536 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f8071f10 0x7fe4f8072cf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80097a0 tx=0x7fe4e802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d479c617bb94f560 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.829+0000 7fe4f559a640 1 -- 172.21.15.103:0/3362706536 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8037070 con 0x7fe4f8071f10 2024-08-19T13:15:01.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.829+0000 7fe4f559a640 1 -- 172.21.15.103:0/3362706536 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e802ae50 con 0x7fe4f8071f10 2024-08-19T13:15:01.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.829+0000 7fe4f559a640 1 -- 172.21.15.103:0/3362706536 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8032660 con 0x7fe4f8071f10 2024-08-19T13:15:01.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.829+0000 7fe4fc9df640 1 -- 172.21.15.103:0/3362706536 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f8071f10 msgr2=0x7fe4f8072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.829+0000 7fe4fc9df640 1 --2- 172.21.15.103:0/3362706536 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f8071f10 0x7fe4f8072cf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80097a0 tx=0x7fe4e802a100 comp rx=0 tx=0).stop 2024-08-19T13:15:01.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.829+0000 7fe4fc9df640 1 -- 172.21.15.103:0/3362706536 shutdown_connections 2024-08-19T13:15:01.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.829+0000 7fe4fc9df640 1 --2- 172.21.15.103:0/3362706536 >> v2:172.21.15.103:3300/0 conn(0x7fe4f8140e80 0x7fe4f813d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.829+0000 7fe4fc9df640 1 --2- 172.21.15.103:0/3362706536 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f8071f10 0x7fe4f8072cf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.829+0000 7fe4fc9df640 1 -- 172.21.15.103:0/3362706536 >> 172.21.15.103:0/3362706536 conn(0x7fe4f80b62f0 msgr2=0x7fe4f80b6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:01.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.829+0000 7fe4fc9df640 1 -- 172.21.15.103:0/3362706536 shutdown_connections 2024-08-19T13:15:01.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.829+0000 7fe4fc9df640 1 -- 172.21.15.103:0/3362706536 wait complete. 2024-08-19T13:15:01.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.830+0000 7fe4fc9df640 1 Processor -- start 2024-08-19T13:15:01.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4fc9df640 1 -- start start 2024-08-19T13:15:01.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4fc9df640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4f8071b30 0x7fe4f8145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4fc9df640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f8071f10 0x7fe4f8146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4fc9df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4f8140e80 0x7fe4f8142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4fc9df640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f813e010 con 0x7fe4f8140e80 2024-08-19T13:15:01.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4fc9df640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f813de90 con 0x7fe4f8071b30 2024-08-19T13:15:01.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4fc9df640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f814ac00 con 0x7fe4f8071f10 2024-08-19T13:15:01.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4f5d9b640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f8071f10 0x7fe4f8146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4f5d9b640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f8071f10 0x7fe4f8146200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48924/0 (socket says 172.21.15.103:48924) 2024-08-19T13:15:01.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4f5d9b640 1 -- 172.21.15.103:0/627895973 learned_addr learned my addr 172.21.15.103:0/627895973 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:01.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4f6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4f8140e80 0x7fe4f8142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4f5d9b640 1 -- 172.21.15.103:0/627895973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4f8071b30 msgr2=0x7fe4f8145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4f5d9b640 1 --2- 172.21.15.103:0/627895973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4f8071b30 0x7fe4f8145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4f5d9b640 1 -- 172.21.15.103:0/627895973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4f8140e80 msgr2=0x7fe4f8142700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:01.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4f5d9b640 1 --2- 172.21.15.103:0/627895973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4f8140e80 0x7fe4f8142700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:01.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4f5d9b640 1 -- 172.21.15.103:0/627895973 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f8147470 con 0x7fe4f8071f10 2024-08-19T13:15:01.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.831+0000 7fe4f6d9d640 1 --2- 172.21.15.103:0/627895973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4f8140e80 0x7fe4f8142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:15:01.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.832+0000 7fe4f5d9b640 1 --2- 172.21.15.103:0/627895973 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f8071f10 0x7fe4f8146200 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80098c0 tx=0x7fe4e80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.832+0000 7fe4e6ffd640 1 -- 172.21.15.103:0/627895973 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8041030 con 0x7fe4f8071f10 2024-08-19T13:15:01.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.832+0000 7fe4e6ffd640 1 -- 172.21.15.103:0/627895973 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e802ac50 con 0x7fe4f8071f10 2024-08-19T13:15:01.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.832+0000 7fe4fc9df640 1 -- 172.21.15.103:0/627895973 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f81476e0 con 0x7fe4f8071f10 2024-08-19T13:15:01.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.832+0000 7fe4e6ffd640 1 -- 172.21.15.103:0/627895973 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8037040 con 0x7fe4f8071f10 2024-08-19T13:15:01.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.832+0000 7fe4fc9df640 1 -- 172.21.15.103:0/627895973 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f8142c40 con 0x7fe4f8071f10 2024-08-19T13:15:01.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.833+0000 7fe4fc9df640 1 -- 172.21.15.103:0/627895973 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f8072cf0 con 0x7fe4f8071f10 2024-08-19T13:15:01.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.833+0000 7fe4e6ffd640 1 -- 172.21.15.103:0/627895973 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe4e8044070 con 0x7fe4f8071f10 2024-08-19T13:15:01.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.834+0000 7fe4e6ffd640 1 --2- 172.21.15.103:0/627895973 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe4d806be30 0x7fe4d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:01.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.834+0000 7fe4f659c640 1 --2- 172.21.15.103:0/627895973 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe4d806be30 0x7fe4d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:01.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.834+0000 7fe4e6ffd640 1 -- 172.21.15.103:0/627895973 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7fe4e80ab270 con 0x7fe4f8071f10 2024-08-19T13:15:01.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.834+0000 7fe4f659c640 1 --2- 172.21.15.103:0/627895973 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe4d806be30 0x7fe4d806e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000f60 tx=0x7fe4ec0079d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:01.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:01.838+0000 7fe4e6ffd640 1 -- 172.21.15.103:0/627895973 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fe4e8079ef0 con 0x7fe4f8071f10 2024-08-19T13:15:02.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.003+0000 7fe4fc9df640 1 -- 172.21.15.103:0/627895973 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4f8146050 con 0x7fe4d806be30 2024-08-19T13:15:02.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.004+0000 7fe4e6ffd640 1 -- 172.21.15.103:0/627895973 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+35912 (secure 0 0 0) 0x7fe4f8146050 con 0x7fe4d806be30 2024-08-19T13:15:02.005 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:15:02.006 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-08-19T13:15:02.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.006+0000 7fe4fc9df640 1 -- 172.21.15.103:0/627895973 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe4d806be30 msgr2=0x7fe4d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.006+0000 7fe4fc9df640 1 --2- 172.21.15.103:0/627895973 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe4d806be30 0x7fe4d806e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000f60 tx=0x7fe4ec0079d0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.006+0000 7fe4fc9df640 1 -- 172.21.15.103:0/627895973 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f8071f10 msgr2=0x7fe4f8146200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.006+0000 7fe4fc9df640 1 --2- 172.21.15.103:0/627895973 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f8071f10 0x7fe4f8146200 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80098c0 tx=0x7fe4e80046c0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.006+0000 7fe4fc9df640 1 -- 172.21.15.103:0/627895973 shutdown_connections 2024-08-19T13:15:02.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.006+0000 7fe4fc9df640 1 --2- 172.21.15.103:0/627895973 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe4d806be30 0x7fe4d806e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.006+0000 7fe4fc9df640 1 --2- 172.21.15.103:0/627895973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4f8140e80 0x7fe4f8142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.006+0000 7fe4fc9df640 1 --2- 172.21.15.103:0/627895973 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f8071f10 0x7fe4f8146200 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.006+0000 7fe4fc9df640 1 --2- 172.21.15.103:0/627895973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4f8071b30 0x7fe4f8145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.006+0000 7fe4fc9df640 1 -- 172.21.15.103:0/627895973 >> 172.21.15.103:0/627895973 conn(0x7fe4f80b62f0 msgr2=0x7fe4f813c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:02.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.006+0000 7fe4fc9df640 1 -- 172.21.15.103:0/627895973 shutdown_connections 2024-08-19T13:15:02.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.006+0000 7fe4fc9df640 1 -- 172.21.15.103:0/627895973 wait complete. 2024-08-19T13:15:02.017 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-08-19T13:15:01.995544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":217752,"kb_used_data":1524,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756968,"statfs":{"total":773094113280,"available":772871135232,"internally_reserved":0,"allocated":1560576,"data_stored":1522318,"data_compressed":58992,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000378"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.168503+0000","last_change":"2024-08-19T13:14:58.168503+0000","last_active":"2024-08-19T13:14:58.168503+0000","last_peered":"2024-08-19T13:14:58.168503+0000","last_clean":"2024-08-19T13:14:58.168503+0000","last_became_active":"2024-08-19T13:14:56.044662+0000","last_became_peered":"2024-08-19T13:14:56.044662+0000","last_unstale":"2024-08-19T13:14:58.168503+0000","last_undegraded":"2024-08-19T13:14:58.168503+0000","last_fullsized":"2024-08-19T13:14:58.168503+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T23:26:23.570412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039371300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.061578+0000","last_change":"2024-08-19T13:14:58.061578+0000","last_active":"2024-08-19T13:14:58.061578+0000","last_peered":"2024-08-19T13:14:58.061578+0000","last_clean":"2024-08-19T13:14:58.061578+0000","last_became_active":"2024-08-19T13:14:56.045731+0000","last_became_peered":"2024-08-19T13:14:56.045731+0000","last_unstale":"2024-08-19T13:14:58.061578+0000","last_undegraded":"2024-08-19T13:14:58.061578+0000","last_fullsized":"2024-08-19T13:14:58.061578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:49:02.156480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044000000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.373877+0000","last_change":"2024-08-19T13:14:58.373877+0000","last_active":"2024-08-19T13:14:58.373877+0000","last_peered":"2024-08-19T13:14:58.373877+0000","last_clean":"2024-08-19T13:14:58.373877+0000","last_became_active":"2024-08-19T13:14:56.111427+0000","last_became_peered":"2024-08-19T13:14:56.111427+0000","last_unstale":"2024-08-19T13:14:58.373877+0000","last_undegraded":"2024-08-19T13:14:58.373877+0000","last_fullsized":"2024-08-19T13:14:58.373877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:19:29.396593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050374000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.061576+0000","last_change":"2024-08-19T13:14:58.061576+0000","last_active":"2024-08-19T13:14:58.061576+0000","last_peered":"2024-08-19T13:14:58.061576+0000","last_clean":"2024-08-19T13:14:58.061576+0000","last_became_active":"2024-08-19T13:14:56.111065+0000","last_became_peered":"2024-08-19T13:14:56.111065+0000","last_unstale":"2024-08-19T13:14:58.061576+0000","last_undegraded":"2024-08-19T13:14:58.061576+0000","last_fullsized":"2024-08-19T13:14:58.061576+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:43:13.276725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042019699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.062290+0000","last_change":"2024-08-19T13:14:58.056552+0000","last_active":"2024-08-19T13:14:58.062290+0000","last_peered":"2024-08-19T13:14:58.062290+0000","last_clean":"2024-08-19T13:14:58.062290+0000","last_became_active":"2024-08-19T13:14:56.045753+0000","last_became_peered":"2024-08-19T13:14:56.045753+0000","last_unstale":"2024-08-19T13:14:58.062290+0000","last_undegraded":"2024-08-19T13:14:58.062290+0000","last_fullsized":"2024-08-19T13:14:58.062290+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:12:21.860038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037158599999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.055759+0000","last_change":"2024-08-19T13:14:58.055759+0000","last_active":"2024-08-19T13:14:58.055759+0000","last_peered":"2024-08-19T13:14:58.055759+0000","last_clean":"2024-08-19T13:14:58.055759+0000","last_became_active":"2024-08-19T13:14:56.046524+0000","last_became_peered":"2024-08-19T13:14:56.046524+0000","last_unstale":"2024-08-19T13:14:58.055759+0000","last_undegraded":"2024-08-19T13:14:58.055759+0000","last_fullsized":"2024-08-19T13:14:58.055759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T23:05:11.782761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043587900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.373682+0000","last_change":"2024-08-19T13:14:58.373682+0000","last_active":"2024-08-19T13:14:58.373682+0000","last_peered":"2024-08-19T13:14:58.373682+0000","last_clean":"2024-08-19T13:14:58.373682+0000","last_became_active":"2024-08-19T13:14:56.045391+0000","last_became_peered":"2024-08-19T13:14:56.045391+0000","last_unstale":"2024-08-19T13:14:58.373682+0000","last_undegraded":"2024-08-19T13:14:58.373682+0000","last_fullsized":"2024-08-19T13:14:58.373682+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:41:33.266324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037776800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.056578+0000","last_change":"2024-08-19T13:14:58.056578+0000","last_active":"2024-08-19T13:14:58.056578+0000","last_peered":"2024-08-19T13:14:58.056578+0000","last_clean":"2024-08-19T13:14:58.056578+0000","last_became_active":"2024-08-19T13:14:56.045464+0000","last_became_peered":"2024-08-19T13:14:56.045464+0000","last_unstale":"2024-08-19T13:14:58.056578+0000","last_undegraded":"2024-08-19T13:14:58.056578+0000","last_fullsized":"2024-08-19T13:14:58.056578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:35:16.028455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039910299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":166,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.373350+0000","last_change":"2024-08-19T13:14:55.043964+0000","last_active":"2024-08-19T13:14:58.373350+0000","last_peered":"2024-08-19T13:14:58.373350+0000","last_clean":"2024-08-19T13:14:58.373350+0000","last_became_active":"2024-08-19T13:14:55.043626+0000","last_became_peered":"2024-08-19T13:14:55.043626+0000","last_unstale":"2024-08-19T13:14:58.373350+0000","last_undegraded":"2024-08-19T13:14:58.373350+0000","last_fullsized":"2024-08-19T13:14:58.373350+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:54.027963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:54.027963+0000","last_clean_scrub_stamp":"2024-08-19T13:14:54.027963+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:36:05.876096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":58992,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":229376,"data_stored":581878,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":58782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":58801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":58782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":58782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":58782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":58801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":587710,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T13:15:02.019 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T13:15:02.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.118+0000 7f7a62fa1640 1 Processor -- start 2024-08-19T13:15:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.119+0000 7f7a62fa1640 1 -- start start 2024-08-19T13:15:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.119+0000 7f7a62fa1640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7a5c0a9d70 0x7f7a5c0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.119+0000 7f7a62fa1640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a5c0aa150 0x7f7a5c0af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.119+0000 7f7a62fa1640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7a5c13f4b0 0x7f7a5c13f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.119+0000 7f7a62fa1640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f7a5c0aae50 con 0x7f7a5c0a9d70 2024-08-19T13:15:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.119+0000 7f7a62fa1640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f7a5c0abd50 con 0x7f7a5c13f4b0 2024-08-19T13:15:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.119+0000 7f7a62fa1640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5c0abfc0 con 0x7f7a5c0aa150 2024-08-19T13:15:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.119+0000 7f7a60d3d640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7a5c0a9d70 0x7f7a5c0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.119+0000 7f7a5bfff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a5c0aa150 0x7f7a5c0af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.119+0000 7f7a60d3d640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7a5c0a9d70 0x7f7a5c0aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50074/0 (socket says 172.21.15.103:50074) 2024-08-19T13:15:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.119+0000 7f7a60d3d640 1 -- 172.21.15.103:0/2050538912 learned_addr learned my addr 172.21.15.103:0/2050538912 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.119+0000 7f7a5bfff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a5c0aa150 0x7f7a5c0af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48926/0 (socket says 172.21.15.103:48926) 2024-08-19T13:15:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.120+0000 7f7a5bfff640 1 -- 172.21.15.103:0/2050538912 >> v2:172.21.15.122:3300/0 conn(0x7f7a5c13f4b0 msgr2=0x7f7a5c13f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:15:02.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.120+0000 7f7a6153e640 1 --2- 172.21.15.103:0/2050538912 >> v2:172.21.15.122:3300/0 conn(0x7f7a5c13f4b0 0x7f7a5c13f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.120+0000 7f7a5bfff640 1 --2- 172.21.15.103:0/2050538912 >> v2:172.21.15.122:3300/0 conn(0x7f7a5c13f4b0 0x7f7a5c13f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.120+0000 7f7a5bfff640 1 -- 172.21.15.103:0/2050538912 >> v2:172.21.15.103:3300/0 conn(0x7f7a5c0a9d70 msgr2=0x7f7a5c0aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:15:02.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.120+0000 7f7a5bfff640 1 --2- 172.21.15.103:0/2050538912 >> v2:172.21.15.103:3300/0 conn(0x7f7a5c0a9d70 0x7f7a5c0aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.120+0000 7f7a5bfff640 1 -- 172.21.15.103:0/2050538912 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c0aa970 con 0x7f7a5c0aa150 2024-08-19T13:15:02.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.120+0000 7f7a60d3d640 1 --2- 172.21.15.103:0/2050538912 >> v2:172.21.15.103:3300/0 conn(0x7f7a5c0a9d70 0x7f7a5c0aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:15:02.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.120+0000 7f7a6153e640 1 --2- 172.21.15.103:0/2050538912 >> v2:172.21.15.122:3300/0 conn(0x7f7a5c13f4b0 0x7f7a5c13f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T13:15:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.120+0000 7f7a5bfff640 1 --2- 172.21.15.103:0/2050538912 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a5c0aa150 0x7f7a5c0af150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0a9950 tx=0x7f7a4802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa38c5b1e1970e41 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.120+0000 7f7a5b7fe640 1 -- 172.21.15.103:0/2050538912 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48037070 con 0x7f7a5c0aa150 2024-08-19T13:15:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.120+0000 7f7a5b7fe640 1 -- 172.21.15.103:0/2050538912 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a48032440 con 0x7f7a5c0aa150 2024-08-19T13:15:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.120+0000 7f7a5b7fe640 1 -- 172.21.15.103:0/2050538912 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4802ae60 con 0x7f7a5c0aa150 2024-08-19T13:15:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.121+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2050538912 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a5c0aa150 msgr2=0x7f7a5c0af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.121+0000 7f7a62fa1640 1 --2- 172.21.15.103:0/2050538912 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a5c0aa150 0x7f7a5c0af150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0a9950 tx=0x7f7a4802a110 comp rx=0 tx=0).stop 2024-08-19T13:15:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.121+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2050538912 shutdown_connections 2024-08-19T13:15:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.121+0000 7f7a62fa1640 1 --2- 172.21.15.103:0/2050538912 >> v2:172.21.15.122:3300/0 conn(0x7f7a5c13f4b0 0x7f7a5c13f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.121+0000 7f7a62fa1640 1 --2- 172.21.15.103:0/2050538912 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a5c0aa150 0x7f7a5c0af150 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.121+0000 7f7a62fa1640 1 --2- 172.21.15.103:0/2050538912 >> v2:172.21.15.103:3300/0 conn(0x7f7a5c0a9d70 0x7f7a5c0aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.121+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2050538912 >> 172.21.15.103:0/2050538912 conn(0x7f7a5c0b4ec0 msgr2=0x7f7a5c0b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.121+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2050538912 shutdown_connections 2024-08-19T13:15:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.121+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2050538912 wait complete. 2024-08-19T13:15:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.122+0000 7f7a62fa1640 1 Processor -- start 2024-08-19T13:15:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.122+0000 7f7a62fa1640 1 -- start start 2024-08-19T13:15:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a62fa1640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a5c0a9d70 0x7f7a5c11cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a62fa1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a5c0aa150 0x7f7a5c1d3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a62fa1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a5c13f4b0 0x7f7a5c1d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a62fa1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c13ffc0 con 0x7f7a5c13f4b0 2024-08-19T13:15:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a62fa1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c13fe40 con 0x7f7a5c0aa150 2024-08-19T13:15:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a62fa1640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5c140140 con 0x7f7a5c0a9d70 2024-08-19T13:15:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a60d3d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a5c0a9d70 0x7f7a5c11cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a6153e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a5c13f4b0 0x7f7a5c1d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a60d3d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a5c0a9d70 0x7f7a5c11cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48930/0 (socket says 172.21.15.103:48930) 2024-08-19T13:15:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a60d3d640 1 -- 172.21.15.103:0/2069217206 learned_addr learned my addr 172.21.15.103:0/2069217206 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a5bfff640 1 --2- 172.21.15.103:0/2069217206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a5c0aa150 0x7f7a5c1d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a6153e640 1 -- 172.21.15.103:0/2069217206 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a5c0a9d70 msgr2=0x7f7a5c11cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a6153e640 1 --2- 172.21.15.103:0/2069217206 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a5c0a9d70 0x7f7a5c11cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a6153e640 1 -- 172.21.15.103:0/2069217206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a5c0aa150 msgr2=0x7f7a5c1d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a6153e640 1 --2- 172.21.15.103:0/2069217206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a5c0aa150 0x7f7a5c1d3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a6153e640 1 -- 172.21.15.103:0/2069217206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c11e1e0 con 0x7f7a5c13f4b0 2024-08-19T13:15:02.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a60d3d640 1 --2- 172.21.15.103:0/2069217206 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a5c0a9d70 0x7f7a5c11cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:15:02.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a6153e640 1 --2- 172.21.15.103:0/2069217206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a5c13f4b0 0x7f7a5c1d3d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400ccc0 tx=0x7f7a5400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:02.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a597fa640 1 -- 172.21.15.103:0/2069217206 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5400f680 con 0x7f7a5c13f4b0 2024-08-19T13:15:02.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.123+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2069217206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5c1d4280 con 0x7f7a5c13f4b0 2024-08-19T13:15:02.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.124+0000 7f7a597fa640 1 -- 172.21.15.103:0/2069217206 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a5400f840 con 0x7f7a5c13f4b0 2024-08-19T13:15:02.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.124+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2069217206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5c0728e0 con 0x7f7a5c13f4b0 2024-08-19T13:15:02.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.124+0000 7f7a597fa640 1 -- 172.21.15.103:0/2069217206 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5400fc30 con 0x7f7a5c13f4b0 2024-08-19T13:15:02.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.125+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2069217206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a5c072c20 con 0x7f7a5c13f4b0 2024-08-19T13:15:02.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.125+0000 7f7a597fa640 1 -- 172.21.15.103:0/2069217206 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a5400e3e0 con 0x7f7a5c13f4b0 2024-08-19T13:15:02.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.125+0000 7f7a597fa640 1 --2- 172.21.15.103:0/2069217206 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7a4006be60 0x7f7a4006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.126+0000 7f7a597fa640 1 -- 172.21.15.103:0/2069217206 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7f7a5408b830 con 0x7f7a5c13f4b0 2024-08-19T13:15:02.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.126+0000 7f7a60d3d640 1 --2- 172.21.15.103:0/2069217206 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7a4006be60 0x7f7a4006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.126+0000 7f7a60d3d640 1 --2- 172.21.15.103:0/2069217206 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7a4006be60 0x7f7a4006e320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000b8d0 tx=0x7f7a5000b450 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:02.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.130+0000 7f7a597fa640 1 -- 172.21.15.103:0/2069217206 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f7a5c072c20 con 0x7f7a5c13f4b0 2024-08-19T13:15:02.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.293+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2069217206 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a5c13f900 con 0x7f7a4006be60 2024-08-19T13:15:02.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.294+0000 7f7a597fa640 1 -- 172.21.15.103:0/2069217206 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+35912 (secure 0 0 0) 0x7f7a5c13f900 con 0x7f7a4006be60 2024-08-19T13:15:02.295 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:15:02.296 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-08-19T13:15:02.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.296+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2069217206 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7a4006be60 msgr2=0x7f7a4006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.296+0000 7f7a62fa1640 1 --2- 172.21.15.103:0/2069217206 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7a4006be60 0x7f7a4006e320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000b8d0 tx=0x7f7a5000b450 comp rx=0 tx=0).stop 2024-08-19T13:15:02.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.296+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2069217206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a5c13f4b0 msgr2=0x7f7a5c1d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.296+0000 7f7a62fa1640 1 --2- 172.21.15.103:0/2069217206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a5c13f4b0 0x7f7a5c1d3d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400ccc0 tx=0x7f7a5400a650 comp rx=0 tx=0).stop 2024-08-19T13:15:02.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.297+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2069217206 shutdown_connections 2024-08-19T13:15:02.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.297+0000 7f7a62fa1640 1 --2- 172.21.15.103:0/2069217206 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7a4006be60 0x7f7a4006e320 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.297+0000 7f7a62fa1640 1 --2- 172.21.15.103:0/2069217206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a5c13f4b0 0x7f7a5c1d3d40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.297+0000 7f7a62fa1640 1 --2- 172.21.15.103:0/2069217206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a5c0aa150 0x7f7a5c1d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.297+0000 7f7a62fa1640 1 --2- 172.21.15.103:0/2069217206 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a5c0a9d70 0x7f7a5c11cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.297+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2069217206 >> 172.21.15.103:0/2069217206 conn(0x7f7a5c0b4ec0 msgr2=0x7f7a5c0aec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:02.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.297+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2069217206 shutdown_connections 2024-08-19T13:15:02.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.297+0000 7f7a62fa1640 1 -- 172.21.15.103:0/2069217206 wait complete. 2024-08-19T13:15:02.308 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-08-19T13:15:01.995544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":217752,"kb_used_data":1524,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756968,"statfs":{"total":773094113280,"available":772871135232,"internally_reserved":0,"allocated":1560576,"data_stored":1522318,"data_compressed":58992,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000378"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.168503+0000","last_change":"2024-08-19T13:14:58.168503+0000","last_active":"2024-08-19T13:14:58.168503+0000","last_peered":"2024-08-19T13:14:58.168503+0000","last_clean":"2024-08-19T13:14:58.168503+0000","last_became_active":"2024-08-19T13:14:56.044662+0000","last_became_peered":"2024-08-19T13:14:56.044662+0000","last_unstale":"2024-08-19T13:14:58.168503+0000","last_undegraded":"2024-08-19T13:14:58.168503+0000","last_fullsized":"2024-08-19T13:14:58.168503+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T23:26:23.570412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039371300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.061578+0000","last_change":"2024-08-19T13:14:58.061578+0000","last_active":"2024-08-19T13:14:58.061578+0000","last_peered":"2024-08-19T13:14:58.061578+0000","last_clean":"2024-08-19T13:14:58.061578+0000","last_became_active":"2024-08-19T13:14:56.045731+0000","last_became_peered":"2024-08-19T13:14:56.045731+0000","last_unstale":"2024-08-19T13:14:58.061578+0000","last_undegraded":"2024-08-19T13:14:58.061578+0000","last_fullsized":"2024-08-19T13:14:58.061578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:49:02.156480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044000000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.373877+0000","last_change":"2024-08-19T13:14:58.373877+0000","last_active":"2024-08-19T13:14:58.373877+0000","last_peered":"2024-08-19T13:14:58.373877+0000","last_clean":"2024-08-19T13:14:58.373877+0000","last_became_active":"2024-08-19T13:14:56.111427+0000","last_became_peered":"2024-08-19T13:14:56.111427+0000","last_unstale":"2024-08-19T13:14:58.373877+0000","last_undegraded":"2024-08-19T13:14:58.373877+0000","last_fullsized":"2024-08-19T13:14:58.373877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:19:29.396593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050374000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.061576+0000","last_change":"2024-08-19T13:14:58.061576+0000","last_active":"2024-08-19T13:14:58.061576+0000","last_peered":"2024-08-19T13:14:58.061576+0000","last_clean":"2024-08-19T13:14:58.061576+0000","last_became_active":"2024-08-19T13:14:56.111065+0000","last_became_peered":"2024-08-19T13:14:56.111065+0000","last_unstale":"2024-08-19T13:14:58.061576+0000","last_undegraded":"2024-08-19T13:14:58.061576+0000","last_fullsized":"2024-08-19T13:14:58.061576+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:43:13.276725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042019699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.062290+0000","last_change":"2024-08-19T13:14:58.056552+0000","last_active":"2024-08-19T13:14:58.062290+0000","last_peered":"2024-08-19T13:14:58.062290+0000","last_clean":"2024-08-19T13:14:58.062290+0000","last_became_active":"2024-08-19T13:14:56.045753+0000","last_became_peered":"2024-08-19T13:14:56.045753+0000","last_unstale":"2024-08-19T13:14:58.062290+0000","last_undegraded":"2024-08-19T13:14:58.062290+0000","last_fullsized":"2024-08-19T13:14:58.062290+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:12:21.860038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037158599999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.055759+0000","last_change":"2024-08-19T13:14:58.055759+0000","last_active":"2024-08-19T13:14:58.055759+0000","last_peered":"2024-08-19T13:14:58.055759+0000","last_clean":"2024-08-19T13:14:58.055759+0000","last_became_active":"2024-08-19T13:14:56.046524+0000","last_became_peered":"2024-08-19T13:14:56.046524+0000","last_unstale":"2024-08-19T13:14:58.055759+0000","last_undegraded":"2024-08-19T13:14:58.055759+0000","last_fullsized":"2024-08-19T13:14:58.055759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T23:05:11.782761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043587900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.373682+0000","last_change":"2024-08-19T13:14:58.373682+0000","last_active":"2024-08-19T13:14:58.373682+0000","last_peered":"2024-08-19T13:14:58.373682+0000","last_clean":"2024-08-19T13:14:58.373682+0000","last_became_active":"2024-08-19T13:14:56.045391+0000","last_became_peered":"2024-08-19T13:14:56.045391+0000","last_unstale":"2024-08-19T13:14:58.373682+0000","last_undegraded":"2024-08-19T13:14:58.373682+0000","last_fullsized":"2024-08-19T13:14:58.373682+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:41:33.266324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037776800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.056578+0000","last_change":"2024-08-19T13:14:58.056578+0000","last_active":"2024-08-19T13:14:58.056578+0000","last_peered":"2024-08-19T13:14:58.056578+0000","last_clean":"2024-08-19T13:14:58.056578+0000","last_became_active":"2024-08-19T13:14:56.045464+0000","last_became_peered":"2024-08-19T13:14:56.045464+0000","last_unstale":"2024-08-19T13:14:58.056578+0000","last_undegraded":"2024-08-19T13:14:58.056578+0000","last_fullsized":"2024-08-19T13:14:58.056578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:35:16.028455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039910299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":166,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T13:14:58.373350+0000","last_change":"2024-08-19T13:14:55.043964+0000","last_active":"2024-08-19T13:14:58.373350+0000","last_peered":"2024-08-19T13:14:58.373350+0000","last_clean":"2024-08-19T13:14:58.373350+0000","last_became_active":"2024-08-19T13:14:55.043626+0000","last_became_peered":"2024-08-19T13:14:55.043626+0000","last_unstale":"2024-08-19T13:14:58.373350+0000","last_undegraded":"2024-08-19T13:14:58.373350+0000","last_fullsized":"2024-08-19T13:14:58.373350+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:54.027963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:54.027963+0000","last_clean_scrub_stamp":"2024-08-19T13:14:54.027963+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:36:05.876096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":58992,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":229376,"data_stored":581878,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":58782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":58801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":58782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":58782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":58782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":58801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":587710,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T13:15:02.308 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-19T13:15:02.309 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-19T13:15:02.309 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-08-19T13:15:02.309 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-08-19T13:15:02.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.406+0000 7ff306c37640 1 Processor -- start 2024-08-19T13:15:02.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.406+0000 7ff306c37640 1 -- start start 2024-08-19T13:15:02.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.407+0000 7ff306c37640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff3000b1fb0 0x7ff300142620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.407+0000 7ff306c37640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3000b2390 0x7ff3000b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.407+0000 7ff306c37640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff300143520 con 0x7ff3000af7e0 2024-08-19T13:15:02.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.407+0000 7ff306c37640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7ff300140270 con 0x7ff3000b1fb0 2024-08-19T13:15:02.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.407+0000 7ff306c37640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff3000afbe0 con 0x7ff3000b2390 2024-08-19T13:15:02.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.407+0000 7ff3051d4640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3000b2390 0x7ff3000b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.407+0000 7ff3051d4640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3000b2390 0x7ff3000b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48934/0 (socket says 172.21.15.103:48934) 2024-08-19T13:15:02.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.407+0000 7ff3051d4640 1 -- 172.21.15.103:0/851578852 learned_addr learned my addr 172.21.15.103:0/851578852 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:02.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.407+0000 7ff3049d3640 1 --2- 172.21.15.103:0/851578852 >> v2:172.21.15.122:3300/0 conn(0x7ff3000b1fb0 0x7ff300142620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.407+0000 7ff3051d4640 1 -- 172.21.15.103:0/851578852 >> v2:172.21.15.122:3300/0 conn(0x7ff3000b1fb0 msgr2=0x7ff300142620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:15:02.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.407+0000 7ff3051d4640 1 --2- 172.21.15.103:0/851578852 >> v2:172.21.15.122:3300/0 conn(0x7ff3000b1fb0 0x7ff300142620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.407+0000 7ff3051d4640 1 -- 172.21.15.103:0/851578852 >> v1:172.21.15.122:6789/0 conn(0x7ff3000af7e0 legacy=0x7ff300142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:15:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.407+0000 7ff3051d4640 1 -- 172.21.15.103:0/851578852 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3000b1820 con 0x7ff3000b2390 2024-08-19T13:15:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.408+0000 7ff3051d4640 1 --2- 172.21.15.103:0/851578852 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3000b2390 0x7ff3000b0070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009e50 tx=0x7ff2f802a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79147a6ffa9478d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.408+0000 7ff2ff7fe640 1 -- 172.21.15.103:0/851578852 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8036070 con 0x7ff3000b2390 2024-08-19T13:15:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.408+0000 7ff2ff7fe640 1 -- 172.21.15.103:0/851578852 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f8031600 con 0x7ff3000b2390 2024-08-19T13:15:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.408+0000 7ff2ff7fe640 1 -- 172.21.15.103:0/851578852 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f80319d0 con 0x7ff3000b2390 2024-08-19T13:15:02.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.409+0000 7ff306c37640 1 -- 172.21.15.103:0/851578852 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3000b2390 msgr2=0x7ff3000b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.409+0000 7ff306c37640 1 --2- 172.21.15.103:0/851578852 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3000b2390 0x7ff3000b0070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009e50 tx=0x7ff2f802a630 comp rx=0 tx=0).stop 2024-08-19T13:15:02.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.409+0000 7ff306c37640 1 -- 172.21.15.103:0/851578852 shutdown_connections 2024-08-19T13:15:02.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.409+0000 7ff306c37640 1 --2- 172.21.15.103:0/851578852 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3000b2390 0x7ff3000b0070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.409+0000 7ff306c37640 1 --2- 172.21.15.103:0/851578852 >> v2:172.21.15.122:3300/0 conn(0x7ff3000b1fb0 0x7ff300142620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.409+0000 7ff306c37640 1 -- 172.21.15.103:0/851578852 >> 172.21.15.103:0/851578852 conn(0x7ff3000b80d0 msgr2=0x7ff3000b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:02.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.409+0000 7ff306c37640 1 -- 172.21.15.103:0/851578852 shutdown_connections 2024-08-19T13:15:02.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.409+0000 7ff306c37640 1 -- 172.21.15.103:0/851578852 wait complete. 2024-08-19T13:15:02.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.410+0000 7ff306c37640 1 Processor -- start 2024-08-19T13:15:02.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.410+0000 7ff306c37640 1 -- start start 2024-08-19T13:15:02.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.410+0000 7ff306c37640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3000af7e0 0x7ff3000a9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.410+0000 7ff306c37640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3000b1fb0 0x7ff3000ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.410+0000 7ff306c37640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3000b2390 0x7ff300149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.410+0000 7ff306c37640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3000b0730 con 0x7ff3000b2390 2024-08-19T13:15:02.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.410+0000 7ff306c37640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff3000b05b0 con 0x7ff3000af7e0 2024-08-19T13:15:02.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.410+0000 7ff306c37640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff30014ce70 con 0x7ff3000b1fb0 2024-08-19T13:15:02.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.410+0000 7ff2fffff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3000b1fb0 0x7ff3000ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.410+0000 7ff3051d4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3000b2390 0x7ff300149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.410+0000 7ff2fffff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3000b1fb0 0x7ff3000ae050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48948/0 (socket says 172.21.15.103:48948) 2024-08-19T13:15:02.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.411+0000 7ff2fffff640 1 -- 172.21.15.103:0/609634745 learned_addr learned my addr 172.21.15.103:0/609634745 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:02.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.411+0000 7ff3049d3640 1 --2- 172.21.15.103:0/609634745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3000af7e0 0x7ff3000a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.411+0000 7ff2fffff640 1 -- 172.21.15.103:0/609634745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3000af7e0 msgr2=0x7ff3000a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.411+0000 7ff2fffff640 1 --2- 172.21.15.103:0/609634745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3000af7e0 0x7ff3000a9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.411+0000 7ff2fffff640 1 -- 172.21.15.103:0/609634745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3000b2390 msgr2=0x7ff300149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.411+0000 7ff2fffff640 1 --2- 172.21.15.103:0/609634745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3000b2390 0x7ff300149040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.411+0000 7ff2fffff640 1 -- 172.21.15.103:0/609634745 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3000aaed0 con 0x7ff3000b1fb0 2024-08-19T13:15:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.411+0000 7ff3051d4640 1 --2- 172.21.15.103:0/609634745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3000b2390 0x7ff300149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:15:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.411+0000 7ff2fffff640 1 --2- 172.21.15.103:0/609634745 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3000b1fb0 0x7ff3000ae050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00c440 tx=0x7ff2ec00c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.411+0000 7ff2fd7fa640 1 -- 172.21.15.103:0/609634745 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec017070 con 0x7ff3000b1fb0 2024-08-19T13:15:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.411+0000 7ff2fd7fa640 1 -- 172.21.15.103:0/609634745 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2ec0035f0 con 0x7ff3000b1fb0 2024-08-19T13:15:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.411+0000 7ff306c37640 1 -- 172.21.15.103:0/609634745 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3000ab1b0 con 0x7ff3000b1fb0 2024-08-19T13:15:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.411+0000 7ff2fd7fa640 1 -- 172.21.15.103:0/609634745 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0039e0 con 0x7ff3000b1fb0 2024-08-19T13:15:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.411+0000 7ff306c37640 1 -- 172.21.15.103:0/609634745 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3000a9ed0 con 0x7ff3000b1fb0 2024-08-19T13:15:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.412+0000 7ff306c37640 1 -- 172.21.15.103:0/609634745 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3000b3fc0 con 0x7ff3000b1fb0 2024-08-19T13:15:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.412+0000 7ff2fd7fa640 1 -- 172.21.15.103:0/609634745 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff2ec01a070 con 0x7ff3000b1fb0 2024-08-19T13:15:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.413+0000 7ff2fd7fa640 1 --2- 172.21.15.103:0/609634745 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff2e406be30 0x7ff2e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.413+0000 7ff2fd7fa640 1 -- 172.21.15.103:0/609634745 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5266+0+0 (secure 0 0 0) 0x7ff2ec089fd0 con 0x7ff3000b1fb0 2024-08-19T13:15:02.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.413+0000 7ff3049d3640 1 --2- 172.21.15.103:0/609634745 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff2e406be30 0x7ff2e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.413+0000 7ff3049d3640 1 --2- 172.21.15.103:0/609634745 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff2e406be30 0x7ff2e406e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400a970 tx=0x7ff2f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:02.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.417+0000 7ff2fd7fa640 1 -- 172.21.15.103:0/609634745 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7ff3000b3fc0 con 0x7ff3000b1fb0 2024-08-19T13:15:02.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.618+0000 7ff306c37640 1 -- 172.21.15.103:0/609634745 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff300142240 con 0x7ff3000b1fb0 2024-08-19T13:15:02.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.618+0000 7ff2fd7fa640 1 -- 172.21.15.103:0/609634745 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff300142240 con 0x7ff3000b1fb0 2024-08-19T13:15:02.619 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:15:02.619 INFO:teuthology.orchestra.run.smithi103.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-19T13:15:02.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.620+0000 7ff306c37640 1 -- 172.21.15.103:0/609634745 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff2e406be30 msgr2=0x7ff2e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.620+0000 7ff306c37640 1 --2- 172.21.15.103:0/609634745 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff2e406be30 0x7ff2e406e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400a970 tx=0x7ff2f4008040 comp rx=0 tx=0).stop 2024-08-19T13:15:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.620+0000 7ff306c37640 1 -- 172.21.15.103:0/609634745 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3000b1fb0 msgr2=0x7ff3000ae050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.620+0000 7ff306c37640 1 --2- 172.21.15.103:0/609634745 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3000b1fb0 0x7ff3000ae050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00c440 tx=0x7ff2ec00c900 comp rx=0 tx=0).stop 2024-08-19T13:15:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.620+0000 7ff306c37640 1 -- 172.21.15.103:0/609634745 shutdown_connections 2024-08-19T13:15:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.620+0000 7ff306c37640 1 --2- 172.21.15.103:0/609634745 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7ff2e406be30 0x7ff2e406e2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.620+0000 7ff306c37640 1 --2- 172.21.15.103:0/609634745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3000b2390 0x7ff300149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.620+0000 7ff306c37640 1 --2- 172.21.15.103:0/609634745 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3000b1fb0 0x7ff3000ae050 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.620+0000 7ff306c37640 1 --2- 172.21.15.103:0/609634745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3000af7e0 0x7ff3000a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.620+0000 7ff306c37640 1 -- 172.21.15.103:0/609634745 >> 172.21.15.103:0/609634745 conn(0x7ff3000b80d0 msgr2=0x7ff300075000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.620+0000 7ff306c37640 1 -- 172.21.15.103:0/609634745 shutdown_connections 2024-08-19T13:15:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.621+0000 7ff306c37640 1 -- 172.21.15.103:0/609634745 wait complete. 2024-08-19T13:15:02.632 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-08-19T13:15:02.632 INFO:teuthology.run_tasks:Running task exec... 2024-08-19T13:15:02.644 INFO:teuthology.task.exec:Executing custom commands... 2024-08-19T13:15:02.644 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T13:15:02.644 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool create cephsqlite' 2024-08-19T13:15:02.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.743+0000 7faac6efe640 1 Processor -- start 2024-08-19T13:15:02.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.743+0000 7faac6efe640 1 -- start start 2024-08-19T13:15:02.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.744+0000 7faac6efe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7faac00aa190 0x7faac00a9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.744+0000 7faac6efe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac013f760 0x7faac013fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.744+0000 7faac6efe640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faac00ab7b0 con 0x7faac00b5400 2024-08-19T13:15:02.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.744+0000 7faac6efe640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7faac00b57e0 con 0x7faac00aa190 2024-08-19T13:15:02.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.744+0000 7faac6efe640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7faac00a9c70 con 0x7faac013f760 2024-08-19T13:15:02.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.744+0000 7faac4c9a640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7faac00b5400 0x7faac01459a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:51176/0 (socket says 172.21.15.103:51176) 2024-08-19T13:15:02.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.744+0000 7faac549b640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac013f760 0x7faac013fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.744+0000 7faac4c9a640 1 -- 172.21.15.103:0/945331767 learned_addr learned my addr 172.21.15.103:0/945331767 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:15:02.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.744+0000 7faabffff640 1 --2- 172.21.15.103:0/945331767 >> v2:172.21.15.122:3300/0 conn(0x7faac00aa190 0x7faac00a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.744+0000 7faac549b640 1 -- 172.21.15.103:0/945331767 >> v2:172.21.15.122:3300/0 conn(0x7faac00aa190 msgr2=0x7faac00a9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:15:02.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.744+0000 7faac549b640 1 --2- 172.21.15.103:0/945331767 >> v2:172.21.15.122:3300/0 conn(0x7faac00aa190 0x7faac00a9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.744+0000 7faac549b640 1 -- 172.21.15.103:0/945331767 >> v1:172.21.15.103:6789/0 conn(0x7faac00b5400 legacy=0x7faac01459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.744+0000 7faabf7fe640 1 -- 172.21.15.103:0/945331767 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 294984349 0 0) 0x7faac00ab7b0 con 0x7faac00b5400 2024-08-19T13:15:02.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.744+0000 7faac549b640 1 -- 172.21.15.103:0/945331767 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac00b4910 con 0x7faac013f760 2024-08-19T13:15:02.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.745+0000 7faac549b640 1 --2- 172.21.15.103:0/945331767 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac013f760 0x7faac013fbb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faab8009d90 tx=0x7faab802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cc9e2eb817c7688 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:02.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.745+0000 7faabf7fe640 1 -- 172.21.15.103:0/945331767 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8007740 con 0x7faac013f760 2024-08-19T13:15:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.745+0000 7faabf7fe640 1 -- 172.21.15.103:0/945331767 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab80078e0 con 0x7faac013f760 2024-08-19T13:15:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.745+0000 7faabf7fe640 1 -- 172.21.15.103:0/945331767 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8007cb0 con 0x7faac013f760 2024-08-19T13:15:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.746+0000 7faac6efe640 1 -- 172.21.15.103:0/945331767 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac013f760 msgr2=0x7faac013fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.746+0000 7faac6efe640 1 --2- 172.21.15.103:0/945331767 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac013f760 0x7faac013fbb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faab8009d90 tx=0x7faab802a660 comp rx=0 tx=0).stop 2024-08-19T13:15:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.746+0000 7faac6efe640 1 -- 172.21.15.103:0/945331767 shutdown_connections 2024-08-19T13:15:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.746+0000 7faac6efe640 1 --2- 172.21.15.103:0/945331767 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac013f760 0x7faac013fbb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.746+0000 7faac6efe640 1 --2- 172.21.15.103:0/945331767 >> v2:172.21.15.122:3300/0 conn(0x7faac00aa190 0x7faac00a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.746+0000 7faac6efe640 1 -- 172.21.15.103:0/945331767 >> 172.21.15.103:0/945331767 conn(0x7faac0075690 msgr2=0x7faac0075aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.746+0000 7faac6efe640 1 -- 172.21.15.103:0/945331767 shutdown_connections 2024-08-19T13:15:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.746+0000 7faac6efe640 1 -- 172.21.15.103:0/945331767 wait complete. 2024-08-19T13:15:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.747+0000 7faac6efe640 1 Processor -- start 2024-08-19T13:15:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.747+0000 7faac6efe640 1 -- start start 2024-08-19T13:15:02.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.747+0000 7faac6efe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faac00aa190 0x7faac0149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.747+0000 7faac6efe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac00b5400 0x7faac0149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.747+0000 7faac6efe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac013f760 0x7faac0146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.747+0000 7faac6efe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faac014ceb0 con 0x7faac013f760 2024-08-19T13:15:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.747+0000 7faac6efe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faac014cd30 con 0x7faac00aa190 2024-08-19T13:15:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.747+0000 7faac6efe640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7faac014d030 con 0x7faac00b5400 2024-08-19T13:15:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.747+0000 7faabffff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac00b5400 0x7faac0149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.747+0000 7faac549b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac013f760 0x7faac0146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.747+0000 7faabffff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac00b5400 0x7faac0149e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48972/0 (socket says 172.21.15.103:48972) 2024-08-19T13:15:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.747+0000 7faabffff640 1 -- 172.21.15.103:0/3253979752 learned_addr learned my addr 172.21.15.103:0/3253979752 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.748+0000 7faac4c9a640 1 --2- 172.21.15.103:0/3253979752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faac00aa190 0x7faac0149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.748+0000 7faabffff640 1 -- 172.21.15.103:0/3253979752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faac00aa190 msgr2=0x7faac0149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.748+0000 7faabffff640 1 --2- 172.21.15.103:0/3253979752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faac00aa190 0x7faac0149870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.748+0000 7faabffff640 1 -- 172.21.15.103:0/3253979752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac013f760 msgr2=0x7faac0146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.748+0000 7faabffff640 1 --2- 172.21.15.103:0/3253979752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac013f760 0x7faac0146ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.748+0000 7faabffff640 1 -- 172.21.15.103:0/3253979752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac014aad0 con 0x7faac00b5400 2024-08-19T13:15:02.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.748+0000 7faac549b640 1 --2- 172.21.15.103:0/3253979752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac013f760 0x7faac0146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:15:02.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.748+0000 7faabffff640 1 --2- 172.21.15.103:0/3253979752 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac00b5400 0x7faac0149e50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faaa800d5f0 tx=0x7faaa800dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:02.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.748+0000 7faabd7fa640 1 -- 172.21.15.103:0/3253979752 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa80125f0 con 0x7faac00b5400 2024-08-19T13:15:02.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.748+0000 7faabd7fa640 1 -- 172.21.15.103:0/3253979752 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa8019070 con 0x7faac00b5400 2024-08-19T13:15:02.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.748+0000 7faac6efe640 1 -- 172.21.15.103:0/3253979752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac014b980 con 0x7faac00b5400 2024-08-19T13:15:02.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.748+0000 7faabd7fa640 1 -- 172.21.15.103:0/3253979752 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa80129d0 con 0x7faac00b5400 2024-08-19T13:15:02.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.748+0000 7faac6efe640 1 -- 172.21.15.103:0/3253979752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faac014a5b0 con 0x7faac00b5400 2024-08-19T13:15:02.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.749+0000 7faac6efe640 1 -- 172.21.15.103:0/3253979752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac00a9670 con 0x7faac00b5400 2024-08-19T13:15:02.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.749+0000 7faabd7fa640 1 -- 172.21.15.103:0/3253979752 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faaa8012b70 con 0x7faac00b5400 2024-08-19T13:15:02.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.750+0000 7faabd7fa640 1 --2- 172.21.15.103:0/3253979752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7faaa006be30 0x7faaa006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:02.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.750+0000 7faac4c9a640 1 --2- 172.21.15.103:0/3253979752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7faaa006be30 0x7faaa006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:02.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.750+0000 7faabd7fa640 1 -- 172.21.15.103:0/3253979752 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 5238+0+0 (secure 0 0 0) 0x7faaa808af00 con 0x7faac00b5400 2024-08-19T13:15:02.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.750+0000 7faac4c9a640 1 --2- 172.21.15.103:0/3253979752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7faaa006be30 0x7faaa006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faab0000f60 tx=0x7faab0007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:02.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.754+0000 7faabd7fa640 1 -- 172.21.15.103:0/3253979752 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7faaa80594c0 con 0x7faac00b5400 2024-08-19T13:15:02.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:02.920+0000 7faac6efe640 1 -- 172.21.15.103:0/3253979752 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephsqlite"} v 0) -- 0x7faac0146840 con 0x7faac00b5400 2024-08-19T13:15:03.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.689+0000 7faabd7fa640 1 -- 172.21.15.103:0/3253979752 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephsqlite"}]=0 pool 'cephsqlite' created v20) ==== 110+0+0 (secure 0 0 0) 0x7faaa805d7c0 con 0x7faac00b5400 2024-08-19T13:15:03.690 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephsqlite' created 2024-08-19T13:15:03.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.693+0000 7faac6efe640 1 -- 172.21.15.103:0/3253979752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7faaa006be30 msgr2=0x7faaa006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:03.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.693+0000 7faac6efe640 1 --2- 172.21.15.103:0/3253979752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7faaa006be30 0x7faaa006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faab0000f60 tx=0x7faab0007d50 comp rx=0 tx=0).stop 2024-08-19T13:15:03.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.693+0000 7faac6efe640 1 -- 172.21.15.103:0/3253979752 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac00b5400 msgr2=0x7faac0149e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:03.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.693+0000 7faac6efe640 1 --2- 172.21.15.103:0/3253979752 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac00b5400 0x7faac0149e50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faaa800d5f0 tx=0x7faaa800dab0 comp rx=0 tx=0).stop 2024-08-19T13:15:03.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.693+0000 7faac6efe640 1 -- 172.21.15.103:0/3253979752 shutdown_connections 2024-08-19T13:15:03.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.693+0000 7faac6efe640 1 --2- 172.21.15.103:0/3253979752 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7faaa006be30 0x7faaa006e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:03.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.693+0000 7faac6efe640 1 --2- 172.21.15.103:0/3253979752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac013f760 0x7faac0146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:03.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.693+0000 7faac6efe640 1 --2- 172.21.15.103:0/3253979752 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac00b5400 0x7faac0149e50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:03.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.693+0000 7faac6efe640 1 --2- 172.21.15.103:0/3253979752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faac00aa190 0x7faac0149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:03.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.693+0000 7faac6efe640 1 -- 172.21.15.103:0/3253979752 >> 172.21.15.103:0/3253979752 conn(0x7faac0075690 msgr2=0x7faac013d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:03.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.693+0000 7faac6efe640 1 -- 172.21.15.103:0/3253979752 shutdown_connections 2024-08-19T13:15:03.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.694+0000 7faac6efe640 1 -- 172.21.15.103:0/3253979752 wait complete. 2024-08-19T13:15:03.706 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph auth get-or-create client.libcephsqlite mon '"'"'profile simple-rados-client-with-blocklist'"'"' osd '"'"'allow rwx pool=cephsqlite'"'"' >> /etc/ceph/ceph.keyring' 2024-08-19T13:15:03.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.808+0000 7f0b35388640 1 Processor -- start 2024-08-19T13:15:03.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.808+0000 7f0b35388640 1 -- start start 2024-08-19T13:15:03.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.809+0000 7f0b35388640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b3013fd00 0x7f0b30146270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:03.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.809+0000 7f0b35388640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b30140130 con 0x7f0b3013f890 2024-08-19T13:15:03.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.809+0000 7f0b35388640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b300b26c0 con 0x7f0b300b52b0 2024-08-19T13:15:03.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.809+0000 7f0b35388640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0b300b5690 con 0x7f0b3013fd00 2024-08-19T13:15:03.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.809+0000 7f0b2e7fc640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f0b3013f890 0x7f0b30145b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:51180/0 (socket says 172.21.15.103:51180) 2024-08-19T13:15:03.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.809+0000 7f0b2e7fc640 1 -- 172.21.15.103:0/3215652344 learned_addr learned my addr 172.21.15.103:0/3215652344 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:15:03.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.809+0000 7f0b2f7fe640 1 --2- 172.21.15.103:0/3215652344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b3013fd00 0x7f0b30146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:03.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.809+0000 7f0b2f7fe640 1 -- 172.21.15.103:0/3215652344 >> v1:172.21.15.122:6789/0 conn(0x7f0b300b52b0 legacy=0x7f0b30142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:03.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.809+0000 7f0b2f7fe640 1 -- 172.21.15.103:0/3215652344 >> v1:172.21.15.103:6789/0 conn(0x7f0b3013f890 legacy=0x7f0b30145b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:03.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.809+0000 7f0b2dffb640 1 -- 172.21.15.103:0/3215652344 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3914270668 0 0) 0x7f0b30140130 con 0x7f0b3013f890 2024-08-19T13:15:03.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.809+0000 7f0b2f7fe640 1 -- 172.21.15.103:0/3215652344 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b300ab830 con 0x7f0b3013fd00 2024-08-19T13:15:03.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.810+0000 7f0b2f7fe640 1 --2- 172.21.15.103:0/3215652344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b3013fd00 0x7f0b30146270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009dc0 tx=0x7f0b2802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fd17a90e4481064 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:03.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.810+0000 7f0b2dffb640 1 -- 172.21.15.103:0/3215652344 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2802ae60 con 0x7f0b3013fd00 2024-08-19T13:15:03.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.810+0000 7f0b2dffb640 1 -- 172.21.15.103:0/3215652344 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b28007740 con 0x7f0b3013fd00 2024-08-19T13:15:03.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.810+0000 7f0b2dffb640 1 -- 172.21.15.103:0/3215652344 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28007b10 con 0x7f0b3013fd00 2024-08-19T13:15:03.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.810+0000 7f0b35388640 1 -- 172.21.15.103:0/3215652344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b3013fd00 msgr2=0x7f0b30146270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:03.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.810+0000 7f0b35388640 1 --2- 172.21.15.103:0/3215652344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b3013fd00 0x7f0b30146270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009dc0 tx=0x7f0b2802a640 comp rx=0 tx=0).stop 2024-08-19T13:15:03.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.811+0000 7f0b35388640 1 -- 172.21.15.103:0/3215652344 shutdown_connections 2024-08-19T13:15:03.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.811+0000 7f0b35388640 1 --2- 172.21.15.103:0/3215652344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b3013fd00 0x7f0b30146270 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:03.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.811+0000 7f0b35388640 1 -- 172.21.15.103:0/3215652344 >> 172.21.15.103:0/3215652344 conn(0x7f0b30075690 msgr2=0x7f0b30075aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:03.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.811+0000 7f0b35388640 1 -- 172.21.15.103:0/3215652344 shutdown_connections 2024-08-19T13:15:03.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.811+0000 7f0b35388640 1 -- 172.21.15.103:0/3215652344 wait complete. 2024-08-19T13:15:03.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.812+0000 7f0b35388640 1 Processor -- start 2024-08-19T13:15:03.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.812+0000 7f0b35388640 1 -- start start 2024-08-19T13:15:03.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.812+0000 7f0b35388640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b300b52b0 0x7f0b300ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:03.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.812+0000 7f0b35388640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b3013f890 0x7f0b300abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:03.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.812+0000 7f0b35388640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b3013fd00 0x7f0b30143880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:03.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.812+0000 7f0b35388640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3014dfa0 con 0x7f0b3013fd00 2024-08-19T13:15:03.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.812+0000 7f0b35388640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3014de20 con 0x7f0b300b52b0 2024-08-19T13:15:03.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.812+0000 7f0b35388640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0b3014e120 con 0x7f0b3013f890 2024-08-19T13:15:03.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.812+0000 7f0b2e7fc640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b3013f890 0x7f0b300abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:03.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.812+0000 7f0b2f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b3013fd00 0x7f0b30143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:03.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.812+0000 7f0b2e7fc640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b3013f890 0x7f0b300abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:48998/0 (socket says 172.21.15.103:48998) 2024-08-19T13:15:03.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.812+0000 7f0b2f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b3013fd00 0x7f0b30143880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50112/0 (socket says 172.21.15.103:50112) 2024-08-19T13:15:03.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.812+0000 7f0b2e7fc640 1 -- 172.21.15.103:0/833373898 learned_addr learned my addr 172.21.15.103:0/833373898 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:15:03.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.813+0000 7f0b2effd640 1 --2- 172.21.15.103:0/833373898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b300b52b0 0x7f0b300ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:03.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.813+0000 7f0b2f7fe640 1 -- 172.21.15.103:0/833373898 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b3013f890 msgr2=0x7f0b300abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:03.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.813+0000 7f0b2f7fe640 1 --2- 172.21.15.103:0/833373898 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b3013f890 0x7f0b300abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:03.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.813+0000 7f0b2f7fe640 1 -- 172.21.15.103:0/833373898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b300b52b0 msgr2=0x7f0b300ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:03.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.813+0000 7f0b2f7fe640 1 --2- 172.21.15.103:0/833373898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b300b52b0 0x7f0b300ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:03.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.813+0000 7f0b2f7fe640 1 -- 172.21.15.103:0/833373898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b300ad950 con 0x7f0b3013fd00 2024-08-19T13:15:03.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.813+0000 7f0b2e7fc640 1 --2- 172.21.15.103:0/833373898 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b3013f890 0x7f0b300abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:15:03.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.813+0000 7f0b2f7fe640 1 --2- 172.21.15.103:0/833373898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b3013fd00 0x7f0b30143880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b28031000 tx=0x7f0b28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:03.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.813+0000 7f0b177fe640 1 -- 172.21.15.103:0/833373898 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28040070 con 0x7f0b3013fd00 2024-08-19T13:15:03.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.813+0000 7f0b177fe640 1 -- 172.21.15.103:0/833373898 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b2802ae60 con 0x7f0b3013fd00 2024-08-19T13:15:03.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.813+0000 7f0b35388640 1 -- 172.21.15.103:0/833373898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b300adc30 con 0x7f0b3013fd00 2024-08-19T13:15:03.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.813+0000 7f0b177fe640 1 -- 172.21.15.103:0/833373898 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28045460 con 0x7f0b3013fd00 2024-08-19T13:15:03.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.813+0000 7f0b35388640 1 -- 172.21.15.103:0/833373898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b300ac3d0 con 0x7f0b3013fd00 2024-08-19T13:15:03.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.814+0000 7f0b35388640 1 -- 172.21.15.103:0/833373898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b300ac600 con 0x7f0b3013fd00 2024-08-19T13:15:03.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.815+0000 7f0b177fe640 1 -- 172.21.15.103:0/833373898 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b28039040 con 0x7f0b3013fd00 2024-08-19T13:15:03.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.815+0000 7f0b177fe640 1 --2- 172.21.15.103:0/833373898 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f0b1c06be30 0x7f0b1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:15:03.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.815+0000 7f0b2effd640 1 --2- 172.21.15.103:0/833373898 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f0b1c06be30 0x7f0b1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:15:03.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.815+0000 7f0b177fe640 1 -- 172.21.15.103:0/833373898 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5559+0+0 (secure 0 0 0) 0x7f0b280ac330 con 0x7f0b3013fd00 2024-08-19T13:15:03.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.816+0000 7f0b2effd640 1 --2- 172.21.15.103:0/833373898 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f0b1c06be30 0x7f0b1c06e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0b20000f60 tx=0x7f0b20009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:15:03.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:03.820+0000 7f0b177fe640 1 -- 172.21.15.103:0/833373898 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f0b2807a870 con 0x7f0b3013fd00 2024-08-19T13:15:04.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:04.051+0000 7f0b35388640 1 -- 172.21.15.103:0/833373898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.libcephsqlite", "caps": ["mon", "profile simple-rados-client-with-blocklist", "osd", "allow rwx pool=cephsqlite"]} v 0) -- 0x7f0b30142a00 con 0x7f0b3013fd00 2024-08-19T13:15:04.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:04.052+0000 7f0b177fe640 1 -- 172.21.15.103:0/833373898 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.libcephsqlite", "caps": ["mon", "profile simple-rados-client-with-blocklist", "osd", "allow rwx pool=cephsqlite"]}]=0 v7) ==== 199+0+71 (secure 0 0 0) 0x7f0b280ac150 con 0x7f0b3013fd00 2024-08-19T13:15:04.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:04.055+0000 7f0b35388640 1 -- 172.21.15.103:0/833373898 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f0b1c06be30 msgr2=0x7f0b1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:04.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:04.055+0000 7f0b35388640 1 --2- 172.21.15.103:0/833373898 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f0b1c06be30 0x7f0b1c06e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0b20000f60 tx=0x7f0b20009040 comp rx=0 tx=0).stop 2024-08-19T13:15:04.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:04.055+0000 7f0b35388640 1 -- 172.21.15.103:0/833373898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b3013fd00 msgr2=0x7f0b30143880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:15:04.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:04.055+0000 7f0b35388640 1 --2- 172.21.15.103:0/833373898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b3013fd00 0x7f0b30143880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b28031000 tx=0x7f0b28004a40 comp rx=0 tx=0).stop 2024-08-19T13:15:04.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:04.055+0000 7f0b35388640 1 -- 172.21.15.103:0/833373898 shutdown_connections 2024-08-19T13:15:04.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:04.055+0000 7f0b35388640 1 --2- 172.21.15.103:0/833373898 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f0b1c06be30 0x7f0b1c06e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:04.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:04.055+0000 7f0b35388640 1 --2- 172.21.15.103:0/833373898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b3013fd00 0x7f0b30143880 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:04.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:04.055+0000 7f0b35388640 1 --2- 172.21.15.103:0/833373898 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b3013f890 0x7f0b300abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:04.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:04.055+0000 7f0b35388640 1 --2- 172.21.15.103:0/833373898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b300b52b0 0x7f0b300ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:15:04.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:04.055+0000 7f0b35388640 1 -- 172.21.15.103:0/833373898 >> 172.21.15.103:0/833373898 conn(0x7f0b30075690 msgr2=0x7f0b3013ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:15:04.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:04.055+0000 7f0b35388640 1 -- 172.21.15.103:0/833373898 shutdown_connections 2024-08-19T13:15:04.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:15:04.055+0000 7f0b35388640 1 -- 172.21.15.103:0/833373898 wait complete. 2024-08-19T13:15:04.066 INFO:teuthology.run_tasks:Running task exec... 2024-08-19T13:15:04.074 INFO:teuthology.task.exec:Executing custom commands... 2024-08-19T13:15:04.075 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T13:15:04.075 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr' 2024-08-19T13:15:04.121 INFO:teuthology.orchestra.run.smithi103.stdout:[==========] Running 20 tests from 1 test suite. 2024-08-19T13:15:04.121 INFO:teuthology.orchestra.run.smithi103.stdout:[----------] Global test environment set-up. 2024-08-19T13:15:04.121 INFO:teuthology.orchestra.run.smithi103.stdout:[----------] 20 tests from CephSQLiteTest 2024-08-19T13:15:09.129 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.Create 2024-08-19T13:15:09.140 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/c4f9b4fe-4ae5-4a0f-a575-d4796b0c1d77.db?vfs=ceph 2024-08-19T13:15:09.140 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:09.144 INFO:teuthology.orchestra.run.smithi103.stdout:CREATE TABLE foo (a INT); 2024-08-19T13:15:09.164 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.Create (35 ms) 2024-08-19T13:15:09.164 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.InsertBulk4096 2024-08-19T13:15:09.174 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/eeb6f3bc-e230-4cc3-8228-9826a1f499e2.db?vfs=ceph 2024-08-19T13:15:09.174 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:09.178 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA page_size = 4096;CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOM() FROM c LIMIT 1000000;PRAGMA page_size; 2024-08-19T13:15:10.834 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.InsertBulk4096 (1670 ms) 2024-08-19T13:15:10.834 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.InsertBulk 2024-08-19T13:15:10.841 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/ae8d9455-02eb-4f30-9f50-782651c02f29.db?vfs=ceph 2024-08-19T13:15:10.842 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:10.848 INFO:teuthology.orchestra.run.smithi103.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOM() FROM c LIMIT 1000000; 2024-08-19T13:15:11.286 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.InsertBulk (452 ms) 2024-08-19T13:15:11.286 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.UpdateBulk 2024-08-19T13:15:11.294 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/2ee6246a-3aff-4f25-b23b-aa2293f21cdb.db?vfs=ceph 2024-08-19T13:15:11.294 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:11.298 INFO:teuthology.orchestra.run.smithi103.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;SELECT SUM(a) FROM foo;UPDATE foo SET a = a+a;SELECT SUM(a) FROM foo; 2024-08-19T13:15:12.014 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.UpdateBulk (728 ms) 2024-08-19T13:15:12.015 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.InsertRate 2024-08-19T13:15:12.022 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/8f9a4aaf-ccfe-4234-8a0f-69ea6208814a.db?vfs=ceph 2024-08-19T13:15:12.022 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:12.025 INFO:teuthology.orchestra.run.smithi103.stdout:CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-08-19T13:15:13.797 INFO:teuthology.orchestra.run.smithi103.stdout:transactions per second: 56.8193 2024-08-19T13:15:13.797 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.InsertRate (1783 ms) 2024-08-19T13:15:13.797 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.DatabaseShrink 2024-08-19T13:15:13.805 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/f1f351fe-d260-4f21-b7e7-63652c7172c0.db?vfs=ceph 2024-08-19T13:15:13.805 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:13.808 INFO:teuthology.orchestra.run.smithi103.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;DELETE FROM foo WHERE RANDOM()%4 < 3;VACUUM; 2024-08-19T13:15:15.109 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.DatabaseShrink (1312 ms) 2024-08-19T13:15:15.109 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveRate 2024-08-19T13:15:15.116 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/a8fc2b4e-29e1-4f22-844a-3a3b27d9f13d.db?vfs=ceph 2024-08-19T13:15:15.117 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:15.120 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA locking_mode=EXCLUSIVE;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-08-19T13:15:15.818 INFO:teuthology.orchestra.run.smithi103.stdout:transactions per second: 145.563 2024-08-19T13:15:15.820 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.InsertExclusiveRate (711 ms) 2024-08-19T13:15:15.820 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveWALRate 2024-08-19T13:15:15.828 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/2214e4d0-dd94-4d54-986b-d1b4a0b7b72f.db?vfs=ceph 2024-08-19T13:15:15.828 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:15.831 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA locking_mode=EXCLUSIVE;PRAGMA journal_mode=WAL;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-08-19T13:15:16.215 INFO:teuthology.orchestra.run.smithi103.stdout:transactions per second: 292.403 2024-08-19T13:15:16.234 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.InsertExclusiveWALRate (414 ms) 2024-08-19T13:15:16.234 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.WALTransactionSync 2024-08-19T13:15:16.241 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/8764497f-2d7d-4d4c-86f5-910a8c943b6b.db?vfs=ceph 2024-08-19T13:15:16.241 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:16.245 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA locking_mode=EXCLUSIVE;PRAGMA journal_mode=WAL;CREATE TABLE foo (a INT);INSERT INTO perf (v) VALUES (ceph_perf());BEGIN TRANSACTION;INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount'; 2024-08-19T13:15:16.303 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.WALTransactionSync (69 ms) 2024-08-19T13:15:16.303 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.PersistTransactionSync 2024-08-19T13:15:16.310 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/d7f4dd7d-1f24-43fb-854a-59d53303fe4c.db?vfs=ceph 2024-08-19T13:15:16.311 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:16.315 INFO:teuthology.orchestra.run.smithi103.stdout:BEGIN TRANSACTION;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount'; 2024-08-19T13:15:16.337 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.PersistTransactionSync (34 ms) 2024-08-19T13:15:16.338 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveLock 2024-08-19T13:15:16.344 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/986e8721-e5bd-4324-aa71-c845ea44aecb.db?vfs=ceph 2024-08-19T13:15:16.344 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:16.347 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA locking_mode=EXCLUSIVE;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_lock.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_lock.avgcount';SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.lock' AND b.fullkey = '$.libcephsqlite_striper.lock'; 2024-08-19T13:15:16.366 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.InsertExclusiveLock (30 ms) 2024-08-19T13:15:16.367 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.TransactionSizeUpdate 2024-08-19T13:15:16.375 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/6f829af3-0791-492b-b9fd-5b35a55a05ae.db?vfs=ceph 2024-08-19T13:15:16.375 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:16.379 INFO:teuthology.orchestra.run.smithi103.stdout:BEGIN TRANSACTION;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.update_size' AND b.fullkey = '$.libcephsqlite_striper.update_size'; 2024-08-19T13:15:16.398 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.TransactionSizeUpdate (31 ms) 2024-08-19T13:15:16.398 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.AllocatedGrowth 2024-08-19T13:15:16.405 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/b5fd9a7b-8022-4cac-bd3f-dc563246ce9d.db?vfs=ceph 2024-08-19T13:15:16.405 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:16.408 INFO:teuthology.orchestra.run.smithi103.stdout:CREATE TABLE foo (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.update_allocated' AND b.fullkey = '$.libcephsqlite_striper.update_allocated'; 2024-08-19T13:15:23.483 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.AllocatedGrowth (7085 ms) 2024-08-19T13:15:23.483 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.DeleteBulk 2024-08-19T13:15:23.490 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/a7621705-7cf6-4e53-8a69-c6e0d9bff22a.db?vfs=ceph 2024-08-19T13:15:23.490 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:23.494 INFO:teuthology.orchestra.run.smithi103.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;DELETE FROM foo WHERE RANDOM()%2 == 0; 2024-08-19T13:15:24.204 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.DeleteBulk (721 ms) 2024-08-19T13:15:24.204 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.DropMassive 2024-08-19T13:15:24.212 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/0c6395d3-cc80-4b58-a1ea-582fa3774ae7.db?vfs=ceph 2024-08-19T13:15:24.212 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:24.216 INFO:teuthology.orchestra.run.smithi103.stdout:CREATE TABLE foo (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;DROP TABLE foo;VACUUM;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.shrink' AND b.fullkey = '$.libcephsqlite_striper.shrink';SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.shrink_bytes' AND b.fullkey = '$.libcephsqlite_striper.shrink_bytes'; 2024-08-19T13:15:53.223 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.DropMassive (29018 ms) 2024-08-19T13:15:53.223 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.InsertMassiveVerify 2024-08-19T13:15:53.231 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/a6ce7099-401d-4052-bcb1-35f0a9f30910.db?vfs=ceph 2024-08-19T13:15:53.231 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:15:53.234 INFO:teuthology.orchestra.run.smithi103.stdout:CREATE TABLE foo (a BLOB);CREATE TEMPORARY TABLE bar (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO bar (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;SELECT a FROM bar;INSERT INTO foo (a) SELECT a FROM bar;SELECT a FROM foo; 2024-08-19T13:16:06.470 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.InsertMassiveVerify (13248 ms) 2024-08-19T13:16:06.470 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.PerfValid 2024-08-19T13:16:06.479 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/417d7535-cebc-45df-ada7-4953a238612f.db?vfs=ceph 2024-08-19T13:16:06.479 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:16:06.486 INFO:teuthology.orchestra.run.smithi103.stdout:SELECT json_valid(ceph_perf()); 2024-08-19T13:16:06.486 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.PerfValid (16 ms) 2024-08-19T13:16:06.486 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.StatusValid 2024-08-19T13:16:06.496 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/3315b2a1-93a8-417e-ab2e-12a01ce39173.db?vfs=ceph 2024-08-19T13:16:06.496 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:16:06.505 INFO:teuthology.orchestra.run.smithi103.stdout:SELECT json_valid(ceph_status()); 2024-08-19T13:16:06.505 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.StatusValid (20 ms) 2024-08-19T13:16:06.505 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.CurrentTime 2024-08-19T13:16:06.515 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/8b291ed7-edbc-47fd-b385-ce6cc95ffef1.db?vfs=ceph 2024-08-19T13:16:06.515 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:16:06.521 INFO:teuthology.orchestra.run.smithi103.stdout:SELECT strftime('%s', 'now'); 2024-08-19T13:16:06.522 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.CurrentTime (16 ms) 2024-08-19T13:16:06.522 INFO:teuthology.orchestra.run.smithi103.stdout:[ RUN ] CephSQLiteTest.StatusFields 2024-08-19T13:16:06.530 INFO:teuthology.orchestra.run.smithi103.stdout:using database: file:cephsqlite:/0bd52776-4e13-4dc8-9abe-7f2ad5f02bd2.db?vfs=ceph 2024-08-19T13:16:06.530 INFO:teuthology.orchestra.run.smithi103.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-08-19T13:16:06.538 INFO:teuthology.orchestra.run.smithi103.stdout:SELECT json_extract(ceph_status(), '$.addr');SELECT json_extract(ceph_status(), '$.id'); 2024-08-19T13:16:06.539 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2603020626 2024-08-19T13:16:06.539 INFO:teuthology.orchestra.run.smithi103.stdout:4434 2024-08-19T13:16:06.539 INFO:teuthology.orchestra.run.smithi103.stdout:[ OK ] CephSQLiteTest.StatusFields (14 ms) 2024-08-19T13:16:06.539 INFO:teuthology.orchestra.run.smithi103.stdout:[----------] 20 tests from CephSQLiteTest (57407 ms total) 2024-08-19T13:16:06.539 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:16:06.539 INFO:teuthology.orchestra.run.smithi103.stdout:[----------] Global test environment tear-down 2024-08-19T13:16:06.539 INFO:teuthology.orchestra.run.smithi103.stdout:[==========] 20 tests from 1 test suite ran. (62415 ms total) 2024-08-19T13:16:06.539 INFO:teuthology.orchestra.run.smithi103.stdout:[ PASSED ] 20 tests. 2024-08-19T13:16:06.577 INFO:teuthology.run_tasks:Running task workunit... 2024-08-19T13:16:06.593 INFO:tasks.workunit:Pulling workunits from ref 78215f5a1cf5eaf321d506a66539942e12851f34 2024-08-19T13:16:06.593 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-08-19T13:16:06.594 DEBUG:teuthology.orchestra.run.smithi103:> stat -- /home/ubuntu/cephtest/mnt.0 2024-08-19T13:16:06.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T13:16:06.609 INFO:teuthology.orchestra.run.smithi103.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-08-19T13:16:06.609 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-08-19T13:16:06.664 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-08-19T13:16:06.665 DEBUG:teuthology.orchestra.run.smithi103:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-08-19T13:16:06.719 INFO:tasks.workunit:timeout=3h 2024-08-19T13:16:06.719 INFO:tasks.workunit:cleanup=True 2024-08-19T13:16:06.720 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 78215f5a1cf5eaf321d506a66539942e12851f34 2024-08-19T13:16:06.775 INFO:tasks.workunit.client.0.smithi103.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-08-19T13:17:06.382 INFO:tasks.workunit.client.0.smithi103.stderr:Note: switching to '78215f5a1cf5eaf321d506a66539942e12851f34'. 2024-08-19T13:17:06.382 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T13:17:06.382 INFO:tasks.workunit.client.0.smithi103.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-08-19T13:17:06.383 INFO:tasks.workunit.client.0.smithi103.stderr:changes and commit them, and you can discard any commits you make in this 2024-08-19T13:17:06.383 INFO:tasks.workunit.client.0.smithi103.stderr:state without impacting any branches by switching back to a branch. 2024-08-19T13:17:06.383 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T13:17:06.383 INFO:tasks.workunit.client.0.smithi103.stderr:If you want to create a new branch to retain commits you create, you may 2024-08-19T13:17:06.383 INFO:tasks.workunit.client.0.smithi103.stderr:do so (now or later) by using -c with the switch command. Example: 2024-08-19T13:17:06.383 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T13:17:06.383 INFO:tasks.workunit.client.0.smithi103.stderr: git switch -c 2024-08-19T13:17:06.383 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T13:17:06.383 INFO:tasks.workunit.client.0.smithi103.stderr:Or undo this operation with: 2024-08-19T13:17:06.383 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T13:17:06.383 INFO:tasks.workunit.client.0.smithi103.stderr: git switch - 2024-08-19T13:17:06.384 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T13:17:06.384 INFO:tasks.workunit.client.0.smithi103.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-08-19T13:17:06.384 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T13:17:06.384 INFO:tasks.workunit.client.0.smithi103.stderr:HEAD is now at 78215f5a1cf Merge pull request #59291 from zdover23/wip-doc-2024-08-19-backport-59290-to-squid 2024-08-19T13:17:06.386 DEBUG:teuthology.orchestra.run.smithi103:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-08-19T13:17:06.449 INFO:tasks.workunit.client.0.smithi103.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-08-19T13:17:06.451 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-19T13:17:06.451 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-08-19T13:17:06.981 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-08-19T13:17:07.053 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-08-19T13:17:07.095 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-19T13:17:07.096 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-19T13:17:07.096 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-08-19T13:17:07.135 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-19T13:17:07.139 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T13:17:07.139 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-08-19T13:17:07.195 INFO:tasks.workunit:Running workunits matching rados/test_libcephsqlite.sh on client.0... 2024-08-19T13:17:07.197 INFO:tasks.workunit:Running workunit rados/test_libcephsqlite.sh... 2024-08-19T13:17:07.197 DEBUG:teuthology.orchestra.run.smithi103:workunit test rados/test_libcephsqlite.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=78215f5a1cf5eaf321d506a66539942e12851f34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CEPH_ARGS='--id libcephsqlite --no-log-to-stderr' adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh cephsqlite 2024-08-19T13:17:07.255 INFO:tasks.workunit.client.0.smithi103.stderr:+ pool=cephsqlite 2024-08-19T13:17:07.255 INFO:tasks.workunit.client.0.smithi103.stderr:++ basename /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh 2024-08-19T13:17:07.256 INFO:tasks.workunit.client.0.smithi103.stderr:+ ns=test_libcephsqlite.sh 2024-08-19T13:17:07.256 INFO:tasks.workunit.client.0.smithi103.stderr:+ striper rm baz.db 2024-08-19T13:17:07.256 INFO:tasks.workunit.client.0.smithi103.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-08-19T13:17:07.300 INFO:tasks.workunit.client.0.smithi103.stderr:+ sqlite 2024-08-19T13:17:07.300 INFO:tasks.workunit.client.0.smithi103.stderr:+ background= 2024-08-19T13:17:07.300 INFO:tasks.workunit.client.0.smithi103.stderr:+ '[' '' = b ']' 2024-08-19T13:17:07.300 INFO:tasks.workunit.client.0.smithi103.stderr:++ cat 2024-08-19T13:17:07.301 INFO:tasks.workunit.client.0.smithi103.stderr:+ a='create table if not exists foo (a INT); 2024-08-19T13:17:07.301 INFO:tasks.workunit.client.0.smithi103.stderr:insert into foo (a) values (RANDOM()); 2024-08-19T13:17:07.301 INFO:tasks.workunit.client.0.smithi103.stderr:drop table foo;' 2024-08-19T13:17:07.301 INFO:tasks.workunit.client.0.smithi103.stderr:+ printf %s 'create table if not exists foo (a INT); 2024-08-19T13:17:07.302 INFO:tasks.workunit.client.0.smithi103.stderr:insert into foo (a) values (RANDOM()); 2024-08-19T13:17:07.302 INFO:tasks.workunit.client.0.smithi103.stderr:drop table foo;' 2024-08-19T13:17:07.302 INFO:tasks.workunit.client.0.smithi103.stderr:create table if not exists foo (a INT); 2024-08-19T13:17:07.302 INFO:tasks.workunit.client.0.smithi103.stderr:insert into foo (a) values (RANDOM()); 2024-08-19T13:17:07.302 INFO:tasks.workunit.client.0.smithi103.stderr:drop table foo;+ '[' '' '!=' b ']' 2024-08-19T13:17:07.302 INFO:tasks.workunit.client.0.smithi103.stderr:+ wait 2024-08-19T13:17:07.302 INFO:tasks.workunit.client.0.smithi103.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-08-19T13:17:07.496 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T13:17:07.496 INFO:tasks.workunit.client.0.smithi103.stderr:real 0m0.196s 2024-08-19T13:17:07.496 INFO:tasks.workunit.client.0.smithi103.stderr:user 0m0.033s 2024-08-19T13:17:07.496 INFO:tasks.workunit.client.0.smithi103.stderr:sys 0m0.024s 2024-08-19T13:17:07.497 INFO:tasks.workunit.client.0.smithi103.stderr:+ striper stat baz.db 2024-08-19T13:17:07.497 INFO:tasks.workunit.client.0.smithi103.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper stat baz.db 2024-08-19T13:17:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:cephsqlite/baz.db mtime 2024-08-19T13:17:07.000000+0000, size 8192 2024-08-19T13:17:07.533 INFO:tasks.workunit.client.0.smithi103.stderr:+ striper rm baz.db 2024-08-19T13:17:07.533 INFO:tasks.workunit.client.0.smithi103.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-08-19T13:17:07.578 INFO:tasks.workunit.client.0.smithi103.stderr:+ sqlite 2024-08-19T13:17:07.578 INFO:tasks.workunit.client.0.smithi103.stderr:++ repeat 10 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:07.578 INFO:tasks.workunit.client.0.smithi103.stderr:++ n=10 2024-08-19T13:17:07.578 INFO:tasks.workunit.client.0.smithi103.stderr:++ shift 2024-08-19T13:17:07.578 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i = 0 )) 2024-08-19T13:17:07.578 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 10 )) 2024-08-19T13:17:07.578 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:07.578 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:07.579 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 10 )) 2024-08-19T13:17:07.579 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:07.579 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:07.579 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 10 )) 2024-08-19T13:17:07.579 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:07.579 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:07.579 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 10 )) 2024-08-19T13:17:07.579 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:07.579 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:07.579 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 10 )) 2024-08-19T13:17:07.579 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:07.579 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:07.579 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 10 )) 2024-08-19T13:17:07.580 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:07.580 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:07.580 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 10 )) 2024-08-19T13:17:07.580 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:07.580 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:07.580 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 10 )) 2024-08-19T13:17:07.580 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:07.580 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:07.580 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 10 )) 2024-08-19T13:17:07.580 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:07.580 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:07.580 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 10 )) 2024-08-19T13:17:07.580 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:07.580 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:07.581 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 10 )) 2024-08-19T13:17:07.581 INFO:tasks.workunit.client.0.smithi103.stderr:+ background= 2024-08-19T13:17:07.581 INFO:tasks.workunit.client.0.smithi103.stderr:+ '[' '' = b ']' 2024-08-19T13:17:07.581 INFO:tasks.workunit.client.0.smithi103.stderr:++ cat 2024-08-19T13:17:07.581 INFO:tasks.workunit.client.0.smithi103.stderr:+ a='CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-08-19T13:17:07.581 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.581 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.581 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.581 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.581 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.581 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.581 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.581 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.581 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.582 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.582 INFO:tasks.workunit.client.0.smithi103.stderr:SELECT LENGTH(text) FROM rand; 2024-08-19T13:17:07.582 INFO:tasks.workunit.client.0.smithi103.stderr:DROP TABLE rand;' 2024-08-19T13:17:07.582 INFO:tasks.workunit.client.0.smithi103.stderr:+ printf %s 'CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-08-19T13:17:07.582 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.582 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.582 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.582 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.582 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.582 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.583 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.583 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.583 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.583 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.583 INFO:tasks.workunit.client.0.smithi103.stderr:SELECT LENGTH(text) FROM rand; 2024-08-19T13:17:07.583 INFO:tasks.workunit.client.0.smithi103.stderr:DROP TABLE rand;' 2024-08-19T13:17:07.584 INFO:tasks.workunit.client.0.smithi103.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-08-19T13:17:07.584 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.584 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.584 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.584 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.585 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.585 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.585 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.585 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.585 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.585 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:07.586 INFO:tasks.workunit.client.0.smithi103.stderr:SELECT LENGTH(text) FROM rand; 2024-08-19T13:17:07.586 INFO:tasks.workunit.client.0.smithi103.stderr:DROP TABLE rand;+ '[' '' '!=' b ']' 2024-08-19T13:17:07.586 INFO:tasks.workunit.client.0.smithi103.stderr:+ wait 2024-08-19T13:17:07.586 INFO:tasks.workunit.client.0.smithi103.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-08-19T13:17:08.187 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.187 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.187 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.187 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.187 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.188 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.188 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.188 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.188 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.188 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.314 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T13:17:08.314 INFO:tasks.workunit.client.0.smithi103.stderr:real 0m0.729s 2024-08-19T13:17:08.315 INFO:tasks.workunit.client.0.smithi103.stderr:user 0m0.080s 2024-08-19T13:17:08.315 INFO:tasks.workunit.client.0.smithi103.stderr:sys 0m0.045s 2024-08-19T13:17:08.315 INFO:tasks.workunit.client.0.smithi103.stderr:+ sqlite 2024-08-19T13:17:08.315 INFO:tasks.workunit.client.0.smithi103.stderr:++ repeat 100 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.315 INFO:tasks.workunit.client.0.smithi103.stderr:++ n=100 2024-08-19T13:17:08.315 INFO:tasks.workunit.client.0.smithi103.stderr:++ shift 2024-08-19T13:17:08.315 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i = 0 )) 2024-08-19T13:17:08.315 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.315 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.315 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.315 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.315 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.316 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.316 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.316 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.316 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.316 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.316 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.316 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.316 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.316 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.316 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.316 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.316 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.316 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.317 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.317 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.317 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.317 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.317 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.317 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.317 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.317 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.317 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.317 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.317 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.317 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.317 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.318 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.318 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.318 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.318 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.318 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.318 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.318 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.318 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.318 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.318 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.318 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.318 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.318 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.319 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.319 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.319 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.319 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.319 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.319 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.319 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.319 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.319 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.319 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.319 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.319 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.319 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.319 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.320 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.321 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.321 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.321 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.321 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.321 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.321 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.321 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.321 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.321 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.321 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.321 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.321 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.322 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.322 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.322 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.322 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.322 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.322 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.322 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.322 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.322 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.322 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.322 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.322 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.322 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.323 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.323 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.323 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.323 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.323 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.323 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.323 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.323 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.323 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.323 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.323 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.323 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.323 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.323 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.324 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.325 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.325 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.325 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.325 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.325 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.325 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.325 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.325 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.325 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.325 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.325 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.325 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.325 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.325 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.326 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.326 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.326 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.326 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.326 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.326 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.326 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.326 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.326 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.326 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.326 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.326 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.326 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.326 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.327 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.327 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.327 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.327 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.327 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.327 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.327 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.327 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.327 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.327 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.327 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.327 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.327 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.327 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.328 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.328 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.328 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.328 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.328 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.328 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.328 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.328 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.328 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.328 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.328 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.328 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.328 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.329 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.330 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.330 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.330 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.330 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.330 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.330 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.330 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.330 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.330 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.330 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.330 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.330 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.330 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.330 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.331 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.331 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.331 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.331 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.331 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.331 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.331 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.331 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.331 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.331 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.331 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.331 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.331 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.331 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.332 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.332 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.332 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.332 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.332 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.332 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.332 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.332 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.332 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.332 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.332 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.332 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.332 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.332 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.333 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.334 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.334 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.334 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.334 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.334 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.334 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.334 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.334 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.334 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.334 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.334 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.334 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.334 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.335 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.335 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.335 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.335 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.335 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.335 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.335 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.335 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.335 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.335 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.335 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.335 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.335 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.335 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.336 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.336 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.336 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.336 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.336 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.336 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.336 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.336 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.336 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.336 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.336 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.336 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.336 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.336 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.337 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.337 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.337 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.337 INFO:tasks.workunit.client.0.smithi103.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-08-19T13:17:08.337 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( ++i )) 2024-08-19T13:17:08.337 INFO:tasks.workunit.client.0.smithi103.stderr:++ (( i < 100 )) 2024-08-19T13:17:08.337 INFO:tasks.workunit.client.0.smithi103.stderr:+ background= 2024-08-19T13:17:08.337 INFO:tasks.workunit.client.0.smithi103.stderr:+ '[' '' = b ']' 2024-08-19T13:17:08.337 INFO:tasks.workunit.client.0.smithi103.stderr:++ cat 2024-08-19T13:17:08.337 INFO:tasks.workunit.client.0.smithi103.stderr:+ a='BEGIN TRANSACTION; 2024-08-19T13:17:08.337 INFO:tasks.workunit.client.0.smithi103.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-08-19T13:17:08.337 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.337 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.337 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.338 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.338 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.338 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.338 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.338 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.338 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.338 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.338 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.338 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.338 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.338 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.338 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.338 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.339 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.339 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.339 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.339 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.339 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.339 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.339 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.339 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.339 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.339 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.339 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.339 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.339 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.339 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.340 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.340 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.340 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.340 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.340 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.340 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.340 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.340 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.340 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.340 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.340 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.340 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.340 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.341 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.341 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.341 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.341 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.341 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.341 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.341 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.341 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.341 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.341 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.341 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.341 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.341 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.341 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.342 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.342 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.342 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.342 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.342 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.342 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.342 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.342 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.342 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.342 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.342 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.342 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.342 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.342 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.343 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.343 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.343 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.343 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.343 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.343 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.343 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.343 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.343 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.343 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.343 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.343 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.343 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.343 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.344 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.344 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.344 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.344 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.344 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.344 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.344 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.344 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.344 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.344 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.344 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.344 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.344 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.344 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:COMMIT; 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:SELECT LENGTH(text) FROM rand; 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:DROP TABLE rand;' 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:+ printf %s 'BEGIN TRANSACTION; 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.345 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.346 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.346 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.346 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.346 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.346 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.346 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.346 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.346 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.346 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.346 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.346 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.346 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.346 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.347 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.347 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.347 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.347 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.347 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.347 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.347 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.347 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.347 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.347 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.347 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.347 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.347 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.347 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.348 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.348 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.348 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.348 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.348 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.348 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.348 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.348 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.348 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.348 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.349 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.349 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.349 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.349 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.349 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.349 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.349 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.349 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.349 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.349 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.349 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.349 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.349 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.349 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.350 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.350 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.350 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.350 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.350 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.350 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.350 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.350 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.350 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.350 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.350 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.350 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.350 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.350 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.351 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.351 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.351 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.351 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.351 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.351 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.351 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.351 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.351 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.351 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.351 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.351 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.351 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.351 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.352 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.352 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.352 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.352 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.352 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.352 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.352 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.352 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.352 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.352 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.352 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.352 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.352 INFO:tasks.workunit.client.0.smithi103.stderr:COMMIT; 2024-08-19T13:17:08.352 INFO:tasks.workunit.client.0.smithi103.stderr:SELECT LENGTH(text) FROM rand; 2024-08-19T13:17:08.353 INFO:tasks.workunit.client.0.smithi103.stderr:DROP TABLE rand;' 2024-08-19T13:17:08.353 INFO:tasks.workunit.client.0.smithi103.stderr:BEGIN TRANSACTION; 2024-08-19T13:17:08.353 INFO:tasks.workunit.client.0.smithi103.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-08-19T13:17:08.353 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.353 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.353 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.353 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.353 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.353 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.353 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.353 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.353 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.353 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.353 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.354 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.354 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.354 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.354 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.354 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.354 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.354 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.354 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.354 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.354 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.354 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.354 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.354 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.355 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.355 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.355 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.355 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.355 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.355 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.355 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.355 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.355 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.355 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.355 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.355 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.355 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.355 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.356 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.356 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.356 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.356 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.356 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.356 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.356 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.356 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.356 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.356 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.356 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.356 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.356 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.356 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.357 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.358 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.358 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.358 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.358 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.358 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.358 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.358 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.358 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.358 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.358 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.358 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.358 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.358 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.358 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.359 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.359 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.359 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.359 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.359 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.359 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.359 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.359 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.359 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.359 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.359 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.359 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.359 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.359 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.360 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.360 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.360 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.360 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.360 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-08-19T13:17:08.360 INFO:tasks.workunit.client.0.smithi103.stderr:COMMIT; 2024-08-19T13:17:08.360 INFO:tasks.workunit.client.0.smithi103.stderr:SELECT LENGTH(text) FROM rand; 2024-08-19T13:17:08.360 INFO:tasks.workunit.client.0.smithi103.stderr:DROP TABLE rand;+ '[' '' '!=' b ']' 2024-08-19T13:17:08.360 INFO:tasks.workunit.client.0.smithi103.stderr:+ wait 2024-08-19T13:17:08.360 INFO:tasks.workunit.client.0.smithi103.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-08-19T13:17:08.541 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.541 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.541 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.541 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.541 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.541 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.541 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.541 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.541 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.541 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.541 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.542 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.542 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.542 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.542 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.542 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.542 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.542 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.542 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.542 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.542 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.543 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.543 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.543 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.543 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.543 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.543 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.543 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.543 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.543 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.543 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.543 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.543 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.543 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.544 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.544 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.544 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.544 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.544 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.544 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.544 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.544 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.544 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.544 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.544 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.545 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.545 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.545 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.545 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.545 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.545 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.545 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.545 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.545 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.545 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.546 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.546 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.546 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.546 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.546 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.546 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.546 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.546 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.546 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.546 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.546 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.546 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.547 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.548 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.548 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.548 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.548 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.548 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.548 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.548 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.548 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.548 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.548 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.548 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.548 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.548 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.548 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.549 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.549 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.549 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:08.549 INFO:tasks.workunit.client.0.smithi103.stdout:4096 2024-08-19T13:17:09.217 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T13:17:09.217 INFO:tasks.workunit.client.0.smithi103.stderr:real 0m0.913s 2024-08-19T13:17:09.217 INFO:tasks.workunit.client.0.smithi103.stderr:user 0m0.067s 2024-08-19T13:17:09.217 INFO:tasks.workunit.client.0.smithi103.stderr:sys 0m0.043s 2024-08-19T13:17:09.217 INFO:tasks.workunit.client.0.smithi103.stderr:+ striper rm baz.db 2024-08-19T13:17:09.217 INFO:tasks.workunit.client.0.smithi103.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-08-19T13:17:09.261 INFO:tasks.workunit.client.0.smithi103.stderr:+ date 2024-08-19T13:17:09.262 INFO:tasks.workunit.client.0.smithi103.stdout:Mon Aug 19 01:17:09 PM UTC 2024 2024-08-19T13:17:09.263 INFO:tasks.workunit.client.0.smithi103.stderr:+ sqlite b 2024-08-19T13:17:09.263 INFO:tasks.workunit.client.0.smithi103.stderr:+ background=b 2024-08-19T13:17:09.263 INFO:tasks.workunit.client.0.smithi103.stderr:+ '[' b = b ']' 2024-08-19T13:17:09.263 INFO:tasks.workunit.client.0.smithi103.stderr:+ shift 2024-08-19T13:17:09.263 INFO:tasks.workunit.client.0.smithi103.stderr:++ cat 2024-08-19T13:17:09.264 INFO:tasks.workunit.client.0.smithi103.stderr:+ a='CREATE TABLE foo (a BLOB); 2024-08-19T13:17:09.264 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO foo VALUES ("start"); 2024-08-19T13:17:09.264 INFO:tasks.workunit.client.0.smithi103.stderr:WITH RECURSIVE c(x) AS 2024-08-19T13:17:09.264 INFO:tasks.workunit.client.0.smithi103.stderr: ( 2024-08-19T13:17:09.264 INFO:tasks.workunit.client.0.smithi103.stderr: VALUES(1) 2024-08-19T13:17:09.264 INFO:tasks.workunit.client.0.smithi103.stderr: UNION ALL 2024-08-19T13:17:09.264 INFO:tasks.workunit.client.0.smithi103.stderr: SELECT x+1 2024-08-19T13:17:09.264 INFO:tasks.workunit.client.0.smithi103.stderr: FROM c 2024-08-19T13:17:09.265 INFO:tasks.workunit.client.0.smithi103.stderr: ) 2024-08-19T13:17:09.265 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO foo (a) 2024-08-19T13:17:09.265 INFO:tasks.workunit.client.0.smithi103.stderr: SELECT RANDOMBLOB(1<<20) 2024-08-19T13:17:09.265 INFO:tasks.workunit.client.0.smithi103.stderr: FROM c 2024-08-19T13:17:09.265 INFO:tasks.workunit.client.0.smithi103.stderr: LIMIT (1<<20);' 2024-08-19T13:17:09.265 INFO:tasks.workunit.client.0.smithi103.stderr:+ printf %s 'CREATE TABLE foo (a BLOB); 2024-08-19T13:17:09.265 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO foo VALUES ("start"); 2024-08-19T13:17:09.265 INFO:tasks.workunit.client.0.smithi103.stderr:WITH RECURSIVE c(x) AS 2024-08-19T13:17:09.265 INFO:tasks.workunit.client.0.smithi103.stderr: ( 2024-08-19T13:17:09.265 INFO:tasks.workunit.client.0.smithi103.stderr: VALUES(1) 2024-08-19T13:17:09.265 INFO:tasks.workunit.client.0.smithi103.stderr: UNION ALL 2024-08-19T13:17:09.265 INFO:tasks.workunit.client.0.smithi103.stderr: SELECT x+1 2024-08-19T13:17:09.265 INFO:tasks.workunit.client.0.smithi103.stderr: FROM c 2024-08-19T13:17:09.266 INFO:tasks.workunit.client.0.smithi103.stderr: ) 2024-08-19T13:17:09.266 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO foo (a) 2024-08-19T13:17:09.266 INFO:tasks.workunit.client.0.smithi103.stderr: SELECT RANDOMBLOB(1<<20) 2024-08-19T13:17:09.266 INFO:tasks.workunit.client.0.smithi103.stderr: FROM c 2024-08-19T13:17:09.266 INFO:tasks.workunit.client.0.smithi103.stderr: LIMIT (1<<20);' 2024-08-19T13:17:09.266 INFO:tasks.workunit.client.0.smithi103.stderr:CREATE TABLE foo (a BLOB); 2024-08-19T13:17:09.266 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO foo VALUES ("start"); 2024-08-19T13:17:09.266 INFO:tasks.workunit.client.0.smithi103.stderr:WITH RECURSIVE c(x) AS 2024-08-19T13:17:09.266 INFO:tasks.workunit.client.0.smithi103.stderr: ( 2024-08-19T13:17:09.266 INFO:tasks.workunit.client.0.smithi103.stderr: VALUES(1) 2024-08-19T13:17:09.266 INFO:tasks.workunit.client.0.smithi103.stderr: UNION ALL 2024-08-19T13:17:09.266 INFO:tasks.workunit.client.0.smithi103.stderr: SELECT x+1 2024-08-19T13:17:09.266 INFO:tasks.workunit.client.0.smithi103.stderr: FROM c 2024-08-19T13:17:09.266 INFO:tasks.workunit.client.0.smithi103.stderr: ) 2024-08-19T13:17:09.267 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO foo (a) 2024-08-19T13:17:09.267 INFO:tasks.workunit.client.0.smithi103.stderr: SELECT RANDOMBLOB(1<<20) 2024-08-19T13:17:09.267 INFO:tasks.workunit.client.0.smithi103.stderr: FROM c 2024-08-19T13:17:09.267 INFO:tasks.workunit.client.0.smithi103.stderr: LIMIT (1<<20);+ '[' b '!=' b ']' 2024-08-19T13:17:09.267 INFO:tasks.workunit.client.0.smithi103.stderr:+ sleep 10 2024-08-19T13:17:09.267 INFO:tasks.workunit.client.0.smithi103.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-08-19T13:17:19.265 INFO:tasks.workunit.client.0.smithi103.stdout:done 2024-08-19T13:17:19.265 INFO:tasks.workunit.client.0.smithi103.stderr:+ echo done 2024-08-19T13:17:19.266 INFO:tasks.workunit.client.0.smithi103.stderr:+ jobs -l 2024-08-19T13:17:19.266 INFO:tasks.workunit.client.0.smithi103.stdout:[1]+ 25639 Running sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" & 2024-08-19T13:17:19.266 INFO:tasks.workunit.client.0.smithi103.stderr:++ jobs -p 2024-08-19T13:17:19.266 INFO:tasks.workunit.client.0.smithi103.stderr:+ kill -KILL -- 25639 2024-08-19T13:17:19.267 INFO:tasks.workunit.client.0.smithi103.stderr:+ date 2024-08-19T13:17:19.267 INFO:tasks.workunit.client.0.smithi103.stdout:Mon Aug 19 01:17:19 PM UTC 2024 2024-08-19T13:17:19.267 INFO:tasks.workunit.client.0.smithi103.stderr:+ wait 2024-08-19T13:17:19.268 INFO:tasks.workunit.client.0.smithi103.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh: line 95: 25639 Killed sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" 2024-08-19T13:17:19.269 INFO:tasks.workunit.client.0.smithi103.stderr:+ date 2024-08-19T13:17:19.269 INFO:tasks.workunit.client.0.smithi103.stdout:Mon Aug 19 01:17:19 PM UTC 2024 2024-08-19T13:17:19.270 INFO:tasks.workunit.client.0.smithi103.stderr:++ sqlite 2024-08-19T13:17:19.270 INFO:tasks.workunit.client.0.smithi103.stderr:++ background= 2024-08-19T13:17:19.270 INFO:tasks.workunit.client.0.smithi103.stderr:++ '[' '' = b ']' 2024-08-19T13:17:19.271 INFO:tasks.workunit.client.0.smithi103.stderr:+++ cat 2024-08-19T13:17:19.272 INFO:tasks.workunit.client.0.smithi103.stderr:++ a='SELECT COUNT(*) FROM foo;' 2024-08-19T13:17:19.272 INFO:tasks.workunit.client.0.smithi103.stderr:++ printf %s 'SELECT COUNT(*) FROM foo;' 2024-08-19T13:17:19.272 INFO:tasks.workunit.client.0.smithi103.stderr:SELECT COUNT(*) FROM foo;++ '[' '' '!=' b ']' 2024-08-19T13:17:19.272 INFO:tasks.workunit.client.0.smithi103.stderr:++ wait 2024-08-19T13:17:19.272 INFO:tasks.workunit.client.0.smithi103.stderr:++ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-08-19T13:17:47.791 INFO:tasks.workunit.client.0.smithi103.stderr:+ n=1 2024-08-19T13:17:47.791 INFO:tasks.workunit.client.0.smithi103.stderr:+ '[' 1 -eq 1 ']' 2024-08-19T13:17:47.791 INFO:tasks.workunit.client.0.smithi103.stderr:+ striper rm baz.db 2024-08-19T13:17:47.791 INFO:tasks.workunit.client.0.smithi103.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-08-19T13:17:47.840 INFO:tasks.workunit.client.0.smithi103.stderr:+ date 2024-08-19T13:17:47.840 INFO:tasks.workunit.client.0.smithi103.stdout:Mon Aug 19 01:17:47 PM UTC 2024 2024-08-19T13:17:47.841 INFO:tasks.workunit.client.0.smithi103.stderr:+ sqlite b 2024-08-19T13:17:47.841 INFO:tasks.workunit.client.0.smithi103.stderr:+ background=b 2024-08-19T13:17:47.841 INFO:tasks.workunit.client.0.smithi103.stderr:+ '[' b = b ']' 2024-08-19T13:17:47.841 INFO:tasks.workunit.client.0.smithi103.stderr:+ shift 2024-08-19T13:17:47.841 INFO:tasks.workunit.client.0.smithi103.stderr:++ cat 2024-08-19T13:17:47.842 INFO:tasks.workunit.client.0.smithi103.stderr:+ a='CREATE TABLE foo (a BLOB); 2024-08-19T13:17:47.842 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO foo VALUES ("start"); 2024-08-19T13:17:47.842 INFO:tasks.workunit.client.0.smithi103.stderr:WITH RECURSIVE c(x) AS 2024-08-19T13:17:47.842 INFO:tasks.workunit.client.0.smithi103.stderr: ( 2024-08-19T13:17:47.842 INFO:tasks.workunit.client.0.smithi103.stderr: VALUES(1) 2024-08-19T13:17:47.842 INFO:tasks.workunit.client.0.smithi103.stderr: UNION ALL 2024-08-19T13:17:47.842 INFO:tasks.workunit.client.0.smithi103.stderr: SELECT x+1 2024-08-19T13:17:47.843 INFO:tasks.workunit.client.0.smithi103.stderr: FROM c 2024-08-19T13:17:47.843 INFO:tasks.workunit.client.0.smithi103.stderr: ) 2024-08-19T13:17:47.843 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO foo (a) 2024-08-19T13:17:47.843 INFO:tasks.workunit.client.0.smithi103.stderr: SELECT RANDOMBLOB(1<<20) 2024-08-19T13:17:47.843 INFO:tasks.workunit.client.0.smithi103.stderr: FROM c 2024-08-19T13:17:47.843 INFO:tasks.workunit.client.0.smithi103.stderr: LIMIT (1<<20);' 2024-08-19T13:17:47.843 INFO:tasks.workunit.client.0.smithi103.stderr:+ printf %s 'CREATE TABLE foo (a BLOB); 2024-08-19T13:17:47.843 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO foo VALUES ("start"); 2024-08-19T13:17:47.843 INFO:tasks.workunit.client.0.smithi103.stderr:WITH RECURSIVE c(x) AS 2024-08-19T13:17:47.843 INFO:tasks.workunit.client.0.smithi103.stderr: ( 2024-08-19T13:17:47.843 INFO:tasks.workunit.client.0.smithi103.stderr: VALUES(1) 2024-08-19T13:17:47.843 INFO:tasks.workunit.client.0.smithi103.stderr: UNION ALL 2024-08-19T13:17:47.843 INFO:tasks.workunit.client.0.smithi103.stderr: SELECT x+1 2024-08-19T13:17:47.843 INFO:tasks.workunit.client.0.smithi103.stderr: FROM c 2024-08-19T13:17:47.844 INFO:tasks.workunit.client.0.smithi103.stderr: ) 2024-08-19T13:17:47.844 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO foo (a) 2024-08-19T13:17:47.844 INFO:tasks.workunit.client.0.smithi103.stderr: SELECT RANDOMBLOB(1<<20) 2024-08-19T13:17:47.844 INFO:tasks.workunit.client.0.smithi103.stderr: FROM c 2024-08-19T13:17:47.844 INFO:tasks.workunit.client.0.smithi103.stderr: LIMIT (1<<20);' 2024-08-19T13:17:47.844 INFO:tasks.workunit.client.0.smithi103.stderr:CREATE TABLE foo (a BLOB); 2024-08-19T13:17:47.844 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO foo VALUES ("start"); 2024-08-19T13:17:47.844 INFO:tasks.workunit.client.0.smithi103.stderr:WITH RECURSIVE c(x) AS 2024-08-19T13:17:47.844 INFO:tasks.workunit.client.0.smithi103.stderr: ( 2024-08-19T13:17:47.844 INFO:tasks.workunit.client.0.smithi103.stderr: VALUES(1) 2024-08-19T13:17:47.844 INFO:tasks.workunit.client.0.smithi103.stderr: UNION ALL 2024-08-19T13:17:47.844 INFO:tasks.workunit.client.0.smithi103.stderr: SELECT x+1 2024-08-19T13:17:47.844 INFO:tasks.workunit.client.0.smithi103.stderr: FROM c 2024-08-19T13:17:47.845 INFO:tasks.workunit.client.0.smithi103.stderr: ) 2024-08-19T13:17:47.845 INFO:tasks.workunit.client.0.smithi103.stderr:INSERT INTO foo (a) 2024-08-19T13:17:47.845 INFO:tasks.workunit.client.0.smithi103.stderr: SELECT RANDOMBLOB(1<<20) 2024-08-19T13:17:47.845 INFO:tasks.workunit.client.0.smithi103.stderr: FROM c 2024-08-19T13:17:47.845 INFO:tasks.workunit.client.0.smithi103.stderr: LIMIT (1<<20);+ '[' b '!=' b ']' 2024-08-19T13:17:47.845 INFO:tasks.workunit.client.0.smithi103.stderr:+ sleep 20 2024-08-19T13:17:47.845 INFO:tasks.workunit.client.0.smithi103.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-08-19T13:18:07.843 INFO:tasks.workunit.client.0.smithi103.stderr:+ jobs -l 2024-08-19T13:18:07.844 INFO:tasks.workunit.client.0.smithi103.stdout:[1]+ 25707 Running sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" & 2024-08-19T13:18:07.845 INFO:tasks.workunit.client.0.smithi103.stderr:++ jobs -p 2024-08-19T13:18:07.845 INFO:tasks.workunit.client.0.smithi103.stderr:+ kill -STOP -- 25707 2024-08-19T13:18:07.845 INFO:tasks.workunit.client.0.smithi103.stderr:+ sleep 45 2024-08-19T13:18:52.845 INFO:tasks.workunit.client.0.smithi103.stderr:+ date 2024-08-19T13:18:52.846 INFO:tasks.workunit.client.0.smithi103.stdout:Mon Aug 19 01:18:52 PM UTC 2024 2024-08-19T13:18:52.847 INFO:tasks.workunit.client.0.smithi103.stderr:++ jobs -p 2024-08-19T13:18:52.847 INFO:tasks.workunit.client.0.smithi103.stderr:+ kill -CONT -- 25707 2024-08-19T13:18:52.847 INFO:tasks.workunit.client.0.smithi103.stderr:+ sleep 10 2024-08-19T13:18:52.864 INFO:tasks.workunit.client.0.smithi103.stderr:Error: near line 3: disk I/O error 2024-08-19T13:18:52.867 INFO:tasks.workunit.client.0.smithi103.stderr:terminate called after throwing an instance of 'std::runtime_error' 2024-08-19T13:18:52.867 INFO:tasks.workunit.client.0.smithi103.stderr: what(): EVP_DecryptInit_ex failed 2024-08-19T13:19:02.848 INFO:tasks.workunit.client.0.smithi103.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh: line 129: 25707 Aborted (core dumped) sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" 2024-08-19T13:19:02.849 INFO:tasks.workunit.client.0.smithi103.stderr:+ date 2024-08-19T13:19:02.849 INFO:tasks.workunit.client.0.smithi103.stdout:Mon Aug 19 01:19:02 PM UTC 2024 2024-08-19T13:19:02.849 INFO:tasks.workunit.client.0.smithi103.stderr:+ wait 2024-08-19T13:19:02.850 INFO:tasks.workunit.client.0.smithi103.stderr:+ date 2024-08-19T13:19:02.850 INFO:tasks.workunit.client.0.smithi103.stdout:Mon Aug 19 01:19:02 PM UTC 2024 2024-08-19T13:19:02.850 INFO:tasks.workunit.client.0.smithi103.stderr:++ sqlite 2024-08-19T13:19:02.850 INFO:tasks.workunit.client.0.smithi103.stderr:++ background= 2024-08-19T13:19:02.851 INFO:tasks.workunit.client.0.smithi103.stderr:++ '[' '' = b ']' 2024-08-19T13:19:02.851 INFO:tasks.workunit.client.0.smithi103.stderr:+++ cat 2024-08-19T13:19:02.851 INFO:tasks.workunit.client.0.smithi103.stderr:++ a='SELECT COUNT(*) FROM foo;' 2024-08-19T13:19:02.851 INFO:tasks.workunit.client.0.smithi103.stderr:++ printf %s 'SELECT COUNT(*) FROM foo;' 2024-08-19T13:19:02.852 INFO:tasks.workunit.client.0.smithi103.stderr:SELECT COUNT(*) FROM foo;++ '[' '' '!=' b ']' 2024-08-19T13:19:02.852 INFO:tasks.workunit.client.0.smithi103.stderr:++ wait 2024-08-19T13:19:02.852 INFO:tasks.workunit.client.0.smithi103.stderr:++ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-08-19T13:19:04.138 INFO:tasks.workunit.client.0.smithi103.stderr:+ n=1 2024-08-19T13:19:04.138 INFO:tasks.workunit.client.0.smithi103.stderr:+ '[' 1 -eq 1 ']' 2024-08-19T13:19:04.139 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-19T13:19:04.139 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-19T13:19:04.206 INFO:tasks.workunit:Stopping ['rados/test_libcephsqlite.sh cephsqlite'] on client.0... 2024-08-19T13:19:04.206 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-08-19T13:19:04.529 DEBUG:teuthology.parallel:result is None 2024-08-19T13:19:04.530 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-08-19T13:19:04.554 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-08-19T13:19:04.555 DEBUG:teuthology.orchestra.run.smithi103:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-08-19T13:19:04.681 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-08-19T13:19:04.681 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-08-19T13:19:04.690 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-19T13:19:04.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T13:19:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.789+0000 7f6ede237640 1 Processor -- start 2024-08-19T13:19:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.789+0000 7f6ede237640 1 -- start start 2024-08-19T13:19:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.790+0000 7f6ede237640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ed81440a0 0x7f6ed8072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.790+0000 7f6ede237640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6ed8144b00 0x7f6ed80a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.790+0000 7f6ede237640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ed8144480 con 0x7f6ed80a94c0 2024-08-19T13:19:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.790+0000 7f6ede237640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed80a99e0 con 0x7f6ed8144b00 2024-08-19T13:19:04.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.790+0000 7f6ede237640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed81418c0 con 0x7f6ed81440a0 2024-08-19T13:19:04.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.790+0000 7f6ed77fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ed81440a0 0x7f6ed8072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:04.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.790+0000 7f6ed77fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ed81440a0 0x7f6ed8072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:50740/0 (socket says 172.21.15.103:50740) 2024-08-19T13:19:04.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.790+0000 7f6ed77fe640 1 -- 172.21.15.103:0/358588602 learned_addr learned my addr 172.21.15.103:0/358588602 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:04.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.790+0000 7f6ed77fe640 1 -- 172.21.15.103:0/358588602 >> v2:172.21.15.122:3300/0 conn(0x7f6ed8144b00 msgr2=0x7f6ed80a9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:04.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.790+0000 7f6ed77fe640 1 --2- 172.21.15.103:0/358588602 >> v2:172.21.15.122:3300/0 conn(0x7f6ed8144b00 0x7f6ed80a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:04.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.790+0000 7f6ed77fe640 1 -- 172.21.15.103:0/358588602 >> v1:172.21.15.122:6789/0 conn(0x7f6ed80a94c0 legacy=0x7f6ed813fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:04.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.790+0000 7f6ed77fe640 1 -- 172.21.15.103:0/358588602 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed8140400 con 0x7f6ed81440a0 2024-08-19T13:19:04.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.791+0000 7f6ed77fe640 1 --2- 172.21.15.103:0/358588602 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ed81440a0 0x7f6ed8072240 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009e50 tx=0x7f6ec802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7d514fe0d58c4c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:04.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.791+0000 7f6ed67fc640 1 -- 172.21.15.103:0/358588602 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8036070 con 0x7f6ed81440a0 2024-08-19T13:19:04.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.791+0000 7f6ed67fc640 1 -- 172.21.15.103:0/358588602 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec8031600 con 0x7f6ed81440a0 2024-08-19T13:19:04.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.791+0000 7f6ed67fc640 1 -- 172.21.15.103:0/358588602 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80319d0 con 0x7f6ed81440a0 2024-08-19T13:19:04.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.792+0000 7f6ede237640 1 -- 172.21.15.103:0/358588602 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ed81440a0 msgr2=0x7f6ed8072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:04.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.792+0000 7f6ede237640 1 --2- 172.21.15.103:0/358588602 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ed81440a0 0x7f6ed8072240 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009e50 tx=0x7f6ec802a660 comp rx=0 tx=0).stop 2024-08-19T13:19:04.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.792+0000 7f6ede237640 1 -- 172.21.15.103:0/358588602 shutdown_connections 2024-08-19T13:19:04.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.792+0000 7f6ede237640 1 --2- 172.21.15.103:0/358588602 >> v2:172.21.15.122:3300/0 conn(0x7f6ed8144b00 0x7f6ed80a9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:04.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.792+0000 7f6ede237640 1 --2- 172.21.15.103:0/358588602 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ed81440a0 0x7f6ed8072240 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:04.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.792+0000 7f6ede237640 1 -- 172.21.15.103:0/358588602 >> 172.21.15.103:0/358588602 conn(0x7f6ed80b3430 msgr2=0x7f6ed80b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:04.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.792+0000 7f6ede237640 1 -- 172.21.15.103:0/358588602 shutdown_connections 2024-08-19T13:19:04.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.792+0000 7f6ede237640 1 -- 172.21.15.103:0/358588602 wait complete. 2024-08-19T13:19:04.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.793+0000 7f6ede237640 1 Processor -- start 2024-08-19T13:19:04.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.793+0000 7f6ede237640 1 -- start start 2024-08-19T13:19:04.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.793+0000 7f6ede237640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ed80a94c0 0x7f6ed8135b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:04.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.793+0000 7f6ede237640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ed81440a0 0x7f6ed8145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:04.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.793+0000 7f6ede237640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ed8144b00 0x7f6ed8130980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:04.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ede237640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed80aa650 con 0x7f6ed81440a0 2024-08-19T13:19:04.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ede237640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed814b1c0 con 0x7f6ed80a94c0 2024-08-19T13:19:04.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ede237640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed81456d0 con 0x7f6ed8144b00 2024-08-19T13:19:04.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ed7fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ed8144b00 0x7f6ed8130980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:04.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ed6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ed81440a0 0x7f6ed8145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:04.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ed7fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ed8144b00 0x7f6ed8130980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:50752/0 (socket says 172.21.15.103:50752) 2024-08-19T13:19:04.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ed6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ed81440a0 0x7f6ed8145190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47434/0 (socket says 172.21.15.103:47434) 2024-08-19T13:19:04.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ed7fff640 1 -- 172.21.15.103:0/2558073228 learned_addr learned my addr 172.21.15.103:0/2558073228 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:04.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ed77fe640 1 --2- 172.21.15.103:0/2558073228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ed80a94c0 0x7f6ed8135b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:04.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ed7fff640 1 -- 172.21.15.103:0/2558073228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ed80a94c0 msgr2=0x7f6ed8135b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:04.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ed7fff640 1 --2- 172.21.15.103:0/2558073228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ed80a94c0 0x7f6ed8135b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:04.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ed7fff640 1 -- 172.21.15.103:0/2558073228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ed81440a0 msgr2=0x7f6ed8145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:04.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ed7fff640 1 --2- 172.21.15.103:0/2558073228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ed81440a0 0x7f6ed8145190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:04.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ed7fff640 1 -- 172.21.15.103:0/2558073228 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed81470b0 con 0x7f6ed8144b00 2024-08-19T13:19:04.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ed6ffd640 1 --2- 172.21.15.103:0/2558073228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ed81440a0 0x7f6ed8145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:04.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ed7fff640 1 --2- 172.21.15.103:0/2558073228 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ed8144b00 0x7f6ed8130980 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000b410 tx=0x7f6ed000b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:04.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ebffff640 1 -- 172.21.15.103:0/2558073228 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0015020 con 0x7f6ed8144b00 2024-08-19T13:19:04.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ebffff640 1 -- 172.21.15.103:0/2558073228 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed0011850 con 0x7f6ed8144b00 2024-08-19T13:19:04.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ede237640 1 -- 172.21.15.103:0/2558073228 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed8145980 con 0x7f6ed8144b00 2024-08-19T13:19:04.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ebffff640 1 -- 172.21.15.103:0/2558073228 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0011c40 con 0x7f6ed8144b00 2024-08-19T13:19:04.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.794+0000 7f6ede237640 1 -- 172.21.15.103:0/2558073228 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed8130ec0 con 0x7f6ed8144b00 2024-08-19T13:19:04.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.795+0000 7f6ede237640 1 -- 172.21.15.103:0/2558073228 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed8072240 con 0x7f6ed8144b00 2024-08-19T13:19:04.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.796+0000 7f6ebffff640 1 -- 172.21.15.103:0/2558073228 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ed0018020 con 0x7f6ed8144b00 2024-08-19T13:19:04.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.796+0000 7f6ebffff640 1 --2- 172.21.15.103:0/2558073228 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f6ec406be60 0x7f6ec406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:04.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.796+0000 7f6ebffff640 1 -- 172.21.15.103:0/2558073228 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7f6ed008a6b0 con 0x7f6ed8144b00 2024-08-19T13:19:04.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.796+0000 7f6ed77fe640 1 --2- 172.21.15.103:0/2558073228 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f6ec406be60 0x7f6ec406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:04.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.796+0000 7f6ed77fe640 1 --2- 172.21.15.103:0/2558073228 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f6ec406be60 0x7f6ec406e320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8033000 tx=0x7f6ec8009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:04.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.801+0000 7f6ebffff640 1 -- 172.21.15.103:0/2558073228 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f6ed001f070 con 0x7f6ed8144b00 2024-08-19T13:19:04.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.966+0000 7f6ede237640 1 -- 172.21.15.103:0/2558073228 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ed8145b50 con 0x7f6ec406be60 2024-08-19T13:19:04.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.970+0000 7f6ebffff640 1 -- 172.21.15.103:0/2558073228 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117854 (secure 0 0 0) 0x7f6ed8145b50 con 0x7f6ec406be60 2024-08-19T13:19:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:19:04.974 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-08-19T13:19:04.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.972+0000 7f6ede237640 1 -- 172.21.15.103:0/2558073228 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f6ec406be60 msgr2=0x7f6ec406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:04.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.972+0000 7f6ede237640 1 --2- 172.21.15.103:0/2558073228 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f6ec406be60 0x7f6ec406e320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8033000 tx=0x7f6ec8009000 comp rx=0 tx=0).stop 2024-08-19T13:19:04.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.972+0000 7f6ede237640 1 -- 172.21.15.103:0/2558073228 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ed8144b00 msgr2=0x7f6ed8130980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:04.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.972+0000 7f6ede237640 1 --2- 172.21.15.103:0/2558073228 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ed8144b00 0x7f6ed8130980 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000b410 tx=0x7f6ed000b810 comp rx=0 tx=0).stop 2024-08-19T13:19:04.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.972+0000 7f6ede237640 1 -- 172.21.15.103:0/2558073228 shutdown_connections 2024-08-19T13:19:04.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.972+0000 7f6ede237640 1 --2- 172.21.15.103:0/2558073228 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f6ec406be60 0x7f6ec406e320 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:04.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.972+0000 7f6ede237640 1 --2- 172.21.15.103:0/2558073228 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ed8144b00 0x7f6ed8130980 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:04.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.972+0000 7f6ede237640 1 --2- 172.21.15.103:0/2558073228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ed81440a0 0x7f6ed8145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:04.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.972+0000 7f6ede237640 1 --2- 172.21.15.103:0/2558073228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ed80a94c0 0x7f6ed8135b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:04.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.972+0000 7f6ede237640 1 -- 172.21.15.103:0/2558073228 >> 172.21.15.103:0/2558073228 conn(0x7f6ed80b3430 msgr2=0x7f6ed80b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:04.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.972+0000 7f6ede237640 1 -- 172.21.15.103:0/2558073228 shutdown_connections 2024-08-19T13:19:04.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:04.972+0000 7f6ede237640 1 -- 172.21.15.103:0/2558073228 wait complete. 2024-08-19T13:19:04.984 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-08-19T13:19:04.047415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3575960891,"num_objects":884,"num_object_clones":0,"num_object_copies":1768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":884,"num_whiteouts":0,"num_read":2623,"num_read_kb":2618,"num_write":162624,"num_write_kb":5650011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":165414,"ondisk_log_size":165414,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9309976,"kb_used_data":6675992,"kb_used_omap":12,"kb_used_meta":2633907,"kb_avail":745664744,"statfs":{"total":773094113280,"available":763560697856,"internally_reserved":0,"allocated":6836215808,"data_stored":6851292878,"data_compressed":70441838,"data_compressed_allocated":73416704,"data_compressed_original":94011392,"omap_allocated":12752,"internal_metadata":2697121328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40960,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002825"},"pg_stats":[{"pgid":"3.1f","version":"22'3551","reported_seq":3577,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.759901+0000","last_change":"2024-08-19T13:15:04.704168+0000","last_active":"2024-08-19T13:17:47.759901+0000","last_peered":"2024-08-19T13:17:47.759901+0000","last_clean":"2024-08-19T13:17:47.759901+0000","last_became_active":"2024-08-19T13:15:04.703897+0000","last_became_peered":"2024-08-19T13:15:04.703897+0000","last_unstale":"2024-08-19T13:17:47.759901+0000","last_undegraded":"2024-08-19T13:17:47.759901+0000","last_fullsized":"2024-08-19T13:17:47.759901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3551,"log_dups_size":0,"ondisk_log_size":3551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:54:14.098742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":3566,"num_write_kb":193154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'5748","reported_seq":5766,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:54.365605+0000","last_change":"2024-08-19T13:15:04.702034+0000","last_active":"2024-08-19T13:17:54.365605+0000","last_peered":"2024-08-19T13:17:54.365605+0000","last_clean":"2024-08-19T13:17:54.365605+0000","last_became_active":"2024-08-19T13:15:04.701653+0000","last_became_peered":"2024-08-19T13:15:04.701653+0000","last_unstale":"2024-08-19T13:17:54.365605+0000","last_undegraded":"2024-08-19T13:17:54.365605+0000","last_fullsized":"2024-08-19T13:17:54.365605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5748,"log_dups_size":0,"ondisk_log_size":5748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T15:23:44.488137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128583504,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5733,"num_write_kb":196004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"23'5874","reported_seq":5890,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:00.011644+0000","last_change":"2024-08-19T13:15:04.708047+0000","last_active":"2024-08-19T13:18:00.011644+0000","last_peered":"2024-08-19T13:18:00.011644+0000","last_clean":"2024-08-19T13:18:00.011644+0000","last_became_active":"2024-08-19T13:15:04.707756+0000","last_became_peered":"2024-08-19T13:15:04.707756+0000","last_unstale":"2024-08-19T13:18:00.011644+0000","last_undegraded":"2024-08-19T13:18:00.011644+0000","last_fullsized":"2024-08-19T13:18:00.011644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5874,"log_dups_size":0,"ondisk_log_size":5874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:23:19.211747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113312264,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5859,"num_write_kb":177354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"22'1998","reported_seq":2013,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.729767+0000","last_change":"2024-08-19T13:15:04.707039+0000","last_active":"2024-08-19T13:17:47.729767+0000","last_peered":"2024-08-19T13:17:47.729767+0000","last_clean":"2024-08-19T13:17:47.729767+0000","last_became_active":"2024-08-19T13:15:04.706726+0000","last_became_peered":"2024-08-19T13:15:04.706726+0000","last_unstale":"2024-08-19T13:17:47.729767+0000","last_undegraded":"2024-08-19T13:17:47.729767+0000","last_fullsized":"2024-08-19T13:17:47.729767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":1998,"log_dups_size":0,"ondisk_log_size":1998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:45:09.075439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1990,"num_write_kb":94721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"23'3781","reported_seq":3797,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.969389+0000","last_change":"2024-08-19T13:15:04.707093+0000","last_active":"2024-08-19T13:17:47.969389+0000","last_peered":"2024-08-19T13:17:47.969389+0000","last_clean":"2024-08-19T13:17:47.969389+0000","last_became_active":"2024-08-19T13:15:04.706832+0000","last_became_peered":"2024-08-19T13:15:04.706832+0000","last_unstale":"2024-08-19T13:17:47.969389+0000","last_undegraded":"2024-08-19T13:17:47.969389+0000","last_fullsized":"2024-08-19T13:17:47.969389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3781,"log_dups_size":0,"ondisk_log_size":3781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T15:22:00.995458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104858112,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3741,"num_write_kb":132744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"23'7047","reported_seq":7066,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:53.256999+0000","last_change":"2024-08-19T13:15:04.706943+0000","last_active":"2024-08-19T13:17:53.256999+0000","last_peered":"2024-08-19T13:17:53.256999+0000","last_clean":"2024-08-19T13:17:53.256999+0000","last_became_active":"2024-08-19T13:15:04.706371+0000","last_became_peered":"2024-08-19T13:15:04.706371+0000","last_unstale":"2024-08-19T13:17:53.256999+0000","last_undegraded":"2024-08-19T13:17:53.256999+0000","last_fullsized":"2024-08-19T13:17:53.256999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":7047,"log_dups_size":0,"ondisk_log_size":7047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:36:21.721116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134217728,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":7029,"num_write_kb":256635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"23'4609","reported_seq":4626,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:04.184395+0000","last_change":"2024-08-19T13:15:04.701266+0000","last_active":"2024-08-19T13:18:04.184395+0000","last_peered":"2024-08-19T13:18:04.184395+0000","last_clean":"2024-08-19T13:18:04.184395+0000","last_became_active":"2024-08-19T13:15:04.700993+0000","last_became_peered":"2024-08-19T13:15:04.700993+0000","last_unstale":"2024-08-19T13:18:04.184395+0000","last_undegraded":"2024-08-19T13:18:04.184395+0000","last_fullsized":"2024-08-19T13:18:04.184395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4609,"log_dups_size":0,"ondisk_log_size":4609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:01:09.023570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109052416,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":4571,"num_write_kb":180994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"23'6372","reported_seq":6387,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:04.058559+0000","last_change":"2024-08-19T13:15:04.699913+0000","last_active":"2024-08-19T13:18:04.058559+0000","last_peered":"2024-08-19T13:18:04.058559+0000","last_clean":"2024-08-19T13:18:04.058559+0000","last_became_active":"2024-08-19T13:15:04.699219+0000","last_became_peered":"2024-08-19T13:15:04.699219+0000","last_unstale":"2024-08-19T13:18:04.058559+0000","last_undegraded":"2024-08-19T13:18:04.058559+0000","last_fullsized":"2024-08-19T13:18:04.058559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":6372,"log_dups_size":0,"ondisk_log_size":6372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:00:52.367696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6366,"num_write_kb":136778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"23'7655","reported_seq":7673,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:07.987425+0000","last_change":"2024-08-19T13:15:04.701157+0000","last_active":"2024-08-19T13:18:07.987425+0000","last_peered":"2024-08-19T13:18:07.987425+0000","last_clean":"2024-08-19T13:18:07.987425+0000","last_became_active":"2024-08-19T13:15:04.700904+0000","last_became_peered":"2024-08-19T13:15:04.700904+0000","last_unstale":"2024-08-19T13:18:07.987425+0000","last_undegraded":"2024-08-19T13:18:07.987425+0000","last_fullsized":"2024-08-19T13:18:07.987425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":7655,"log_dups_size":0,"ondisk_log_size":7655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:41:45.337705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146932240,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":7658,"num_write_kb":230670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"23'3889","reported_seq":3904,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.917748+0000","last_change":"2024-08-19T13:15:04.704857+0000","last_active":"2024-08-19T13:17:47.917748+0000","last_peered":"2024-08-19T13:17:47.917748+0000","last_clean":"2024-08-19T13:17:47.917748+0000","last_became_active":"2024-08-19T13:15:04.704383+0000","last_became_peered":"2024-08-19T13:15:04.704383+0000","last_unstale":"2024-08-19T13:17:47.917748+0000","last_undegraded":"2024-08-19T13:17:47.917748+0000","last_fullsized":"2024-08-19T13:17:47.917748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3889,"log_dups_size":0,"ondisk_log_size":3889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:27:48.271684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106366632,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3864,"num_write_kb":162549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"23'8294","reported_seq":8332,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:03.337243+0000","last_change":"2024-08-19T13:15:04.703573+0000","last_active":"2024-08-19T13:18:03.337243+0000","last_peered":"2024-08-19T13:18:03.337243+0000","last_clean":"2024-08-19T13:18:03.337243+0000","last_became_active":"2024-08-19T13:15:04.703277+0000","last_became_peered":"2024-08-19T13:15:04.703277+0000","last_unstale":"2024-08-19T13:18:03.337243+0000","last_undegraded":"2024-08-19T13:18:03.337243+0000","last_fullsized":"2024-08-19T13:18:03.337243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":8294,"log_dups_size":0,"ondisk_log_size":8294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:08:33.844086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125960192,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":85,"num_read_kb":132,"num_write":8326,"num_write_kb":182642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"23'6457","reported_seq":6573,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:57.290424+0000","last_change":"2024-08-19T13:15:04.707221+0000","last_active":"2024-08-19T13:17:57.290424+0000","last_peered":"2024-08-19T13:17:57.290424+0000","last_clean":"2024-08-19T13:17:57.290424+0000","last_became_active":"2024-08-19T13:15:04.706496+0000","last_became_peered":"2024-08-19T13:15:04.706496+0000","last_unstale":"2024-08-19T13:17:57.290424+0000","last_undegraded":"2024-08-19T13:17:57.290424+0000","last_fullsized":"2024-08-19T13:17:57.290424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":6457,"log_dups_size":0,"ondisk_log_size":6457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:43:34.377186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90964496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":6419,"num_write_kb":175583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.730287+0000","last_change":"2024-08-19T13:14:58.168503+0000","last_active":"2024-08-19T13:17:47.730287+0000","last_peered":"2024-08-19T13:17:47.730287+0000","last_clean":"2024-08-19T13:17:47.730287+0000","last_became_active":"2024-08-19T13:14:56.044662+0000","last_became_peered":"2024-08-19T13:14:56.044662+0000","last_unstale":"2024-08-19T13:17:47.730287+0000","last_undegraded":"2024-08-19T13:17:47.730287+0000","last_fullsized":"2024-08-19T13:17:47.730287+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T23:26:23.570412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039371300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'6792","reported_seq":6816,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:09.561558+0000","last_change":"2024-08-19T13:15:04.700796+0000","last_active":"2024-08-19T13:18:09.561558+0000","last_peered":"2024-08-19T13:18:09.561558+0000","last_clean":"2024-08-19T13:18:09.561558+0000","last_became_active":"2024-08-19T13:15:04.700540+0000","last_became_peered":"2024-08-19T13:15:04.700540+0000","last_unstale":"2024-08-19T13:18:09.561558+0000","last_undegraded":"2024-08-19T13:18:09.561558+0000","last_fullsized":"2024-08-19T13:18:09.561558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":6792,"log_dups_size":0,"ondisk_log_size":6792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:39:03.153240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104988672,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":34,"num_read_kb":151,"num_write":6787,"num_write_kb":141259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.758067+0000","last_change":"2024-08-19T13:14:58.061578+0000","last_active":"2024-08-19T13:17:47.758067+0000","last_peered":"2024-08-19T13:17:47.758067+0000","last_clean":"2024-08-19T13:17:47.758067+0000","last_became_active":"2024-08-19T13:14:56.045731+0000","last_became_peered":"2024-08-19T13:14:56.045731+0000","last_unstale":"2024-08-19T13:17:47.758067+0000","last_undegraded":"2024-08-19T13:17:47.758067+0000","last_fullsized":"2024-08-19T13:17:47.758067+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:49:02.156480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044000000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'5379","reported_seq":5402,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.968669+0000","last_change":"2024-08-19T13:15:04.703333+0000","last_active":"2024-08-19T13:17:47.968669+0000","last_peered":"2024-08-19T13:17:47.968669+0000","last_clean":"2024-08-19T13:17:47.968669+0000","last_became_active":"2024-08-19T13:15:04.703011+0000","last_became_peered":"2024-08-19T13:15:04.703011+0000","last_unstale":"2024-08-19T13:17:47.968669+0000","last_undegraded":"2024-08-19T13:17:47.968669+0000","last_fullsized":"2024-08-19T13:17:47.968669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5379,"log_dups_size":0,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:08:52.537445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146276352,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5343,"num_write_kb":213509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.740138+0000","last_change":"2024-08-19T13:14:58.373877+0000","last_active":"2024-08-19T13:17:47.740138+0000","last_peered":"2024-08-19T13:17:47.740138+0000","last_clean":"2024-08-19T13:17:47.740138+0000","last_became_active":"2024-08-19T13:14:56.111427+0000","last_became_peered":"2024-08-19T13:14:56.111427+0000","last_unstale":"2024-08-19T13:17:47.740138+0000","last_undegraded":"2024-08-19T13:17:47.740138+0000","last_fullsized":"2024-08-19T13:17:47.740138+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:19:29.396593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050374000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'8281","reported_seq":8296,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:06.692281+0000","last_change":"2024-08-19T13:15:04.699221+0000","last_active":"2024-08-19T13:18:06.692281+0000","last_peered":"2024-08-19T13:18:06.692281+0000","last_clean":"2024-08-19T13:18:06.692281+0000","last_became_active":"2024-08-19T13:15:04.698944+0000","last_became_peered":"2024-08-19T13:15:04.698944+0000","last_unstale":"2024-08-19T13:18:06.692281+0000","last_undegraded":"2024-08-19T13:18:06.692281+0000","last_fullsized":"2024-08-19T13:18:06.692281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":8281,"log_dups_size":0,"ondisk_log_size":8281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:45:55.451611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8278,"num_write_kb":279553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.757967+0000","last_change":"2024-08-19T13:14:58.061576+0000","last_active":"2024-08-19T13:17:47.757967+0000","last_peered":"2024-08-19T13:17:47.757967+0000","last_clean":"2024-08-19T13:17:47.757967+0000","last_became_active":"2024-08-19T13:14:56.111065+0000","last_became_peered":"2024-08-19T13:14:56.111065+0000","last_unstale":"2024-08-19T13:17:47.757967+0000","last_undegraded":"2024-08-19T13:17:47.757967+0000","last_fullsized":"2024-08-19T13:17:47.757967+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:43:13.276725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042019699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'4283","reported_seq":4312,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:52.878133+0000","last_change":"2024-08-19T13:15:04.698014+0000","last_active":"2024-08-19T13:18:52.878133+0000","last_peered":"2024-08-19T13:18:52.878133+0000","last_clean":"2024-08-19T13:18:52.878133+0000","last_became_active":"2024-08-19T13:15:04.697675+0000","last_became_peered":"2024-08-19T13:15:04.697675+0000","last_unstale":"2024-08-19T13:18:52.878133+0000","last_undegraded":"2024-08-19T13:18:52.878133+0000","last_fullsized":"2024-08-19T13:18:52.878133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4283,"log_dups_size":0,"ondisk_log_size":4283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:28:29.123627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114253824,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":58,"num_read_kb":45,"num_write":4306,"num_write_kb":159578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'5377","reported_seq":5407,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.733133+0000","last_change":"2024-08-19T13:15:04.703723+0000","last_active":"2024-08-19T13:17:47.733133+0000","last_peered":"2024-08-19T13:17:47.733133+0000","last_clean":"2024-08-19T13:17:47.733133+0000","last_became_active":"2024-08-19T13:15:04.703417+0000","last_became_peered":"2024-08-19T13:15:04.703417+0000","last_unstale":"2024-08-19T13:17:47.733133+0000","last_undegraded":"2024-08-19T13:17:47.733133+0000","last_fullsized":"2024-08-19T13:17:47.733133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5377,"log_dups_size":0,"ondisk_log_size":5377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:11:03.088099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":5405,"num_write_kb":165854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.729507+0000","last_change":"2024-08-19T13:14:58.056552+0000","last_active":"2024-08-19T13:17:47.729507+0000","last_peered":"2024-08-19T13:17:47.729507+0000","last_clean":"2024-08-19T13:17:47.729507+0000","last_became_active":"2024-08-19T13:14:56.045753+0000","last_became_peered":"2024-08-19T13:14:56.045753+0000","last_unstale":"2024-08-19T13:17:47.729507+0000","last_undegraded":"2024-08-19T13:17:47.729507+0000","last_fullsized":"2024-08-19T13:17:47.729507+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:12:21.860038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037158599999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"23'2823","reported_seq":2843,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.965052+0000","last_change":"2024-08-19T13:15:04.699545+0000","last_active":"2024-08-19T13:17:47.965052+0000","last_peered":"2024-08-19T13:17:47.965052+0000","last_clean":"2024-08-19T13:17:47.965052+0000","last_became_active":"2024-08-19T13:15:04.699255+0000","last_became_peered":"2024-08-19T13:15:04.699255+0000","last_unstale":"2024-08-19T13:17:47.965052+0000","last_undegraded":"2024-08-19T13:17:47.965052+0000","last_fullsized":"2024-08-19T13:17:47.965052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":2823,"log_dups_size":0,"ondisk_log_size":2823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:46:06.725740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84017152,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":2784,"num_write_kb":135939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.736581+0000","last_change":"2024-08-19T13:14:58.055759+0000","last_active":"2024-08-19T13:17:47.736581+0000","last_peered":"2024-08-19T13:17:47.736581+0000","last_clean":"2024-08-19T13:17:47.736581+0000","last_became_active":"2024-08-19T13:14:56.046524+0000","last_became_peered":"2024-08-19T13:14:56.046524+0000","last_unstale":"2024-08-19T13:17:47.736581+0000","last_undegraded":"2024-08-19T13:17:47.736581+0000","last_fullsized":"2024-08-19T13:17:47.736581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T23:05:11.782761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043587900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"23'5930","reported_seq":5946,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.915146+0000","last_change":"2024-08-19T13:15:04.710024+0000","last_active":"2024-08-19T13:17:47.915146+0000","last_peered":"2024-08-19T13:17:47.915146+0000","last_clean":"2024-08-19T13:17:47.915146+0000","last_became_active":"2024-08-19T13:15:04.709768+0000","last_became_peered":"2024-08-19T13:15:04.709768+0000","last_unstale":"2024-08-19T13:17:47.915146+0000","last_undegraded":"2024-08-19T13:17:47.915146+0000","last_fullsized":"2024-08-19T13:17:47.915146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5930,"log_dups_size":0,"ondisk_log_size":5930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:26:22.787995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146866696,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5915,"num_write_kb":254104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.739748+0000","last_change":"2024-08-19T13:14:58.373682+0000","last_active":"2024-08-19T13:17:47.739748+0000","last_peered":"2024-08-19T13:17:47.739748+0000","last_clean":"2024-08-19T13:17:47.739748+0000","last_became_active":"2024-08-19T13:14:56.045391+0000","last_became_peered":"2024-08-19T13:14:56.045391+0000","last_unstale":"2024-08-19T13:17:47.739748+0000","last_undegraded":"2024-08-19T13:17:47.739748+0000","last_fullsized":"2024-08-19T13:17:47.739748+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:41:33.266324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037776800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"23'5375","reported_seq":5407,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:01.600150+0000","last_change":"2024-08-19T13:15:04.698787+0000","last_active":"2024-08-19T13:18:01.600150+0000","last_peered":"2024-08-19T13:18:01.600150+0000","last_clean":"2024-08-19T13:18:01.600150+0000","last_became_active":"2024-08-19T13:15:04.698015+0000","last_became_peered":"2024-08-19T13:15:04.698015+0000","last_unstale":"2024-08-19T13:18:01.600150+0000","last_undegraded":"2024-08-19T13:18:01.600150+0000","last_fullsized":"2024-08-19T13:18:01.600150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5375,"log_dups_size":0,"ondisk_log_size":5375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:07:36.161135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":68,"num_read_kb":114,"num_write":5405,"num_write_kb":156434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.729454+0000","last_change":"2024-08-19T13:14:58.056578+0000","last_active":"2024-08-19T13:17:47.729454+0000","last_peered":"2024-08-19T13:17:47.729454+0000","last_clean":"2024-08-19T13:17:47.729454+0000","last_became_active":"2024-08-19T13:14:56.045464+0000","last_became_peered":"2024-08-19T13:14:56.045464+0000","last_unstale":"2024-08-19T13:17:47.729454+0000","last_undegraded":"2024-08-19T13:17:47.729454+0000","last_fullsized":"2024-08-19T13:17:47.729454+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:35:16.028455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039910299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.739769+0000","last_change":"2024-08-19T13:14:55.043964+0000","last_active":"2024-08-19T13:17:47.739769+0000","last_peered":"2024-08-19T13:17:47.739769+0000","last_clean":"2024-08-19T13:17:47.739769+0000","last_became_active":"2024-08-19T13:14:55.043626+0000","last_became_peered":"2024-08-19T13:14:55.043626+0000","last_unstale":"2024-08-19T13:17:47.739769+0000","last_undegraded":"2024-08-19T13:17:47.739769+0000","last_fullsized":"2024-08-19T13:17:47.739769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:54.027963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:54.027963+0000","last_clean_scrub_stamp":"2024-08-19T13:14:54.027963+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:36:05.876096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"23'2767","reported_seq":2782,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.965472+0000","last_change":"2024-08-19T13:15:04.699203+0000","last_active":"2024-08-19T13:17:47.965472+0000","last_peered":"2024-08-19T13:17:47.965472+0000","last_clean":"2024-08-19T13:17:47.965472+0000","last_became_active":"2024-08-19T13:15:04.698903+0000","last_became_peered":"2024-08-19T13:15:04.698903+0000","last_unstale":"2024-08-19T13:17:47.965472+0000","last_undegraded":"2024-08-19T13:17:47.965472+0000","last_fullsized":"2024-08-19T13:17:47.965472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:32:06.151208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2699,"num_write_kb":115587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"23'9933","reported_seq":10062,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:04.629618+0000","last_change":"2024-08-19T13:15:04.708930+0000","last_active":"2024-08-19T13:18:04.629618+0000","last_peered":"2024-08-19T13:18:04.629618+0000","last_clean":"2024-08-19T13:18:04.629618+0000","last_became_active":"2024-08-19T13:15:04.706442+0000","last_became_peered":"2024-08-19T13:15:04.706442+0000","last_unstale":"2024-08-19T13:18:04.629618+0000","last_undegraded":"2024-08-19T13:18:04.629618+0000","last_fullsized":"2024-08-19T13:18:04.629618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":9933,"log_dups_size":0,"ondisk_log_size":9933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:18:30.218698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145424384,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":361,"num_read_kb":322,"num_write":6867,"num_write_kb":212554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"23'5336","reported_seq":5354,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:05.747745+0000","last_change":"2024-08-19T13:15:04.708013+0000","last_active":"2024-08-19T13:18:05.747745+0000","last_peered":"2024-08-19T13:18:05.747745+0000","last_clean":"2024-08-19T13:18:05.747745+0000","last_became_active":"2024-08-19T13:15:04.707724+0000","last_became_peered":"2024-08-19T13:15:04.707724+0000","last_unstale":"2024-08-19T13:18:05.747745+0000","last_undegraded":"2024-08-19T13:18:05.747745+0000","last_fullsized":"2024-08-19T13:18:05.747745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:59:28.795104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":11,"num_read_kb":71,"num_write":5419,"num_write_kb":161478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"22'3547","reported_seq":3562,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.729697+0000","last_change":"2024-08-19T13:15:04.700346+0000","last_active":"2024-08-19T13:17:47.729697+0000","last_peered":"2024-08-19T13:17:47.729697+0000","last_clean":"2024-08-19T13:17:47.729697+0000","last_became_active":"2024-08-19T13:15:04.699228+0000","last_became_peered":"2024-08-19T13:15:04.699228+0000","last_unstale":"2024-08-19T13:17:47.729697+0000","last_undegraded":"2024-08-19T13:17:47.729697+0000","last_fullsized":"2024-08-19T13:17:47.729697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3547,"log_dups_size":0,"ondisk_log_size":3547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T23:29:48.186111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3536,"num_write_kb":144645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"23'3397","reported_seq":3441,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.967033+0000","last_change":"2024-08-19T13:15:04.699676+0000","last_active":"2024-08-19T13:17:47.967033+0000","last_peered":"2024-08-19T13:17:47.967033+0000","last_clean":"2024-08-19T13:17:47.967033+0000","last_became_active":"2024-08-19T13:15:04.699383+0000","last_became_peered":"2024-08-19T13:15:04.699383+0000","last_unstale":"2024-08-19T13:17:47.967033+0000","last_undegraded":"2024-08-19T13:17:47.967033+0000","last_fullsized":"2024-08-19T13:17:47.967033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3397,"log_dups_size":0,"ondisk_log_size":3397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:10:11.285767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":121,"num_read_kb":99,"num_write":3411,"num_write_kb":138057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'4376","reported_seq":4398,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.916399+0000","last_change":"2024-08-19T13:15:04.709245+0000","last_active":"2024-08-19T13:17:47.916399+0000","last_peered":"2024-08-19T13:17:47.916399+0000","last_clean":"2024-08-19T13:17:47.916399+0000","last_became_active":"2024-08-19T13:15:04.708174+0000","last_became_peered":"2024-08-19T13:15:04.708174+0000","last_unstale":"2024-08-19T13:17:47.916399+0000","last_undegraded":"2024-08-19T13:17:47.916399+0000","last_fullsized":"2024-08-19T13:17:47.916399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4376,"log_dups_size":0,"ondisk_log_size":4376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:25:57.887618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123076608,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":27,"num_read_kb":82,"num_write":4364,"num_write_kb":209707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'4908","reported_seq":5001,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:48.004070+0000","last_change":"2024-08-19T13:15:04.707739+0000","last_active":"2024-08-19T13:17:48.004070+0000","last_peered":"2024-08-19T13:17:48.004070+0000","last_clean":"2024-08-19T13:17:48.004070+0000","last_became_active":"2024-08-19T13:15:04.707482+0000","last_became_peered":"2024-08-19T13:15:04.707482+0000","last_unstale":"2024-08-19T13:17:48.004070+0000","last_undegraded":"2024-08-19T13:17:48.004070+0000","last_fullsized":"2024-08-19T13:17:48.004070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4908,"log_dups_size":0,"ondisk_log_size":4908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:07:38.674944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123349040,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":288,"num_read_kb":216,"num_write":5188,"num_write_kb":204415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'3580","reported_seq":3595,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.969742+0000","last_change":"2024-08-19T13:15:04.707186+0000","last_active":"2024-08-19T13:17:47.969742+0000","last_peered":"2024-08-19T13:17:47.969742+0000","last_clean":"2024-08-19T13:17:47.969742+0000","last_became_active":"2024-08-19T13:15:04.706891+0000","last_became_peered":"2024-08-19T13:15:04.706891+0000","last_unstale":"2024-08-19T13:17:47.969742+0000","last_undegraded":"2024-08-19T13:17:47.969742+0000","last_fullsized":"2024-08-19T13:17:47.969742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3580,"log_dups_size":0,"ondisk_log_size":3580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:45:10.115661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114560672,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3532,"num_write_kb":190257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"23'2893","reported_seq":2908,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.966115+0000","last_change":"2024-08-19T13:15:04.704399+0000","last_active":"2024-08-19T13:17:47.966115+0000","last_peered":"2024-08-19T13:17:47.966115+0000","last_clean":"2024-08-19T13:17:47.966115+0000","last_became_active":"2024-08-19T13:15:04.704113+0000","last_became_peered":"2024-08-19T13:15:04.704113+0000","last_unstale":"2024-08-19T13:17:47.966115+0000","last_undegraded":"2024-08-19T13:17:47.966115+0000","last_fullsized":"2024-08-19T13:17:47.966115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":2893,"log_dups_size":0,"ondisk_log_size":2893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:09:37.265840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103415808,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2845,"num_write_kb":157315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"22'4138","reported_seq":4158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.739708+0000","last_change":"2024-08-19T13:15:04.709668+0000","last_active":"2024-08-19T13:17:47.739708+0000","last_peered":"2024-08-19T13:17:47.739708+0000","last_clean":"2024-08-19T13:17:47.739708+0000","last_became_active":"2024-08-19T13:15:04.709376+0000","last_became_peered":"2024-08-19T13:15:04.709376+0000","last_unstale":"2024-08-19T13:17:47.739708+0000","last_undegraded":"2024-08-19T13:17:47.739708+0000","last_fullsized":"2024-08-19T13:17:47.739708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:26:41.219376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":4141,"num_write_kb":181515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"23'3049","reported_seq":3079,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.966674+0000","last_change":"2024-08-19T13:15:04.704226+0000","last_active":"2024-08-19T13:17:47.966674+0000","last_peered":"2024-08-19T13:17:47.966674+0000","last_clean":"2024-08-19T13:17:47.966674+0000","last_became_active":"2024-08-19T13:15:04.703965+0000","last_became_peered":"2024-08-19T13:15:04.703965+0000","last_unstale":"2024-08-19T13:17:47.966674+0000","last_undegraded":"2024-08-19T13:17:47.966674+0000","last_fullsized":"2024-08-19T13:17:47.966674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3049,"log_dups_size":0,"ondisk_log_size":3049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:51:52.289558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79700496,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":58,"num_read_kb":46,"num_write":3047,"num_write_kb":132115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"23'7855","reported_seq":8198,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:59.958424+0000","last_change":"2024-08-19T13:15:04.707304+0000","last_active":"2024-08-19T13:17:59.958424+0000","last_peered":"2024-08-19T13:17:59.958424+0000","last_clean":"2024-08-19T13:17:59.958424+0000","last_became_active":"2024-08-19T13:15:04.706952+0000","last_became_peered":"2024-08-19T13:15:04.706952+0000","last_unstale":"2024-08-19T13:17:59.958424+0000","last_undegraded":"2024-08-19T13:17:59.958424+0000","last_fullsized":"2024-08-19T13:17:59.958424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":7855,"log_dups_size":0,"ondisk_log_size":7855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:27:03.627133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1110,"num_read_kb":992,"num_write":8069,"num_write_kb":173890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3575370504,"num_objects":880,"num_object_clones":0,"num_object_copies":1760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":880,"num_whiteouts":0,"num_read":2507,"num_read_kb":2518,"num_write":162463,"num_write_kb":5647593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6834151424,"data_stored":6849530008,"data_compressed":70382846,"data_compressed_allocated":73334784,"data_compressed_original":92987392,"omap_allocated":0,"internal_metadata":0},"log_size":165294,"ondisk_log_size":165294,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":58992,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1448372,"kb_used_data":1016044,"kb_used_omap":1,"kb_used_meta":432318,"kb_avail":92923468,"statfs":{"total":96636764160,"available":95153631232,"internally_reserved":0,"allocated":1040429056,"data_stored":1041943846,"data_compressed":9088911,"data_compressed_allocated":9474048,"data_compressed_original":11792384,"omap_allocated":1590,"internal_metadata":442694090},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443108,"kb_used_data":335580,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":93928732,"statfs":{"total":96636764160,"available":96183021568,"internally_reserved":0,"allocated":343633920,"data_stored":344904662,"data_compressed":5573327,"data_compressed_allocated":5816320,"data_compressed_original":7417856,"omap_allocated":1594,"internal_metadata":110098886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1250856,"kb_used_data":930976,"kb_used_omap":1,"kb_used_meta":319870,"kb_avail":93120984,"statfs":{"total":96636764160,"available":95355887616,"internally_reserved":0,"allocated":953319424,"data_stored":957661653,"data_compressed":10069671,"data_compressed_allocated":10522624,"data_compressed_original":15568896,"omap_allocated":1595,"internal_metadata":327547333},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1865984,"kb_used_data":1357368,"kb_used_omap":1,"kb_used_meta":508606,"kb_avail":92505856,"statfs":{"total":96636764160,"available":94725996544,"internally_reserved":0,"allocated":1389944832,"data_stored":1392673978,"data_compressed":17106045,"data_compressed_allocated":17801216,"data_compressed_original":21614592,"omap_allocated":1595,"internal_metadata":520812997},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1480504,"kb_used_data":1056496,"kb_used_omap":1,"kb_used_meta":423998,"kb_avail":92891336,"statfs":{"total":96636764160,"available":95120728064,"internally_reserved":0,"allocated":1081851904,"data_stored":1083347674,"data_compressed":11134625,"data_compressed_allocated":11599872,"data_compressed_original":13860864,"omap_allocated":1593,"internal_metadata":434174407},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496100,"kb_used_data":338652,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":93875740,"statfs":{"total":96636764160,"available":96128757760,"internally_reserved":0,"allocated":346779648,"data_stored":346707202,"data_compressed":1288748,"data_compressed_allocated":1335296,"data_compressed_original":1609728,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":874888,"kb_used_data":600832,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":93496952,"statfs":{"total":96636764160,"available":95740878848,"internally_reserved":0,"allocated":615251968,"data_stored":615348941,"data_compressed":3731208,"data_compressed_allocated":3866624,"data_compressed_original":4513792,"omap_allocated":1595,"internal_metadata":280623557},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1450164,"kb_used_data":1040044,"kb_used_omap":1,"kb_used_meta":410110,"kb_avail":92921676,"statfs":{"total":96636764160,"available":95151796224,"internally_reserved":0,"allocated":1065005056,"data_stored":1068704922,"data_compressed":12449303,"data_compressed_allocated":13000704,"data_compressed_original":17633280,"omap_allocated":1595,"internal_metadata":419953093},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1064701952,"data_stored":1068087872,"data_compressed":12419807,"data_compressed_allocated":12959744,"data_compressed_original":17121280,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":615006208,"data_stored":615260800,"data_compressed":3731208,"data_compressed_allocated":3866624,"data_compressed_original":4513792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346537984,"data_stored":346619080,"data_compressed":1288748,"data_compressed_allocated":1335296,"data_compressed_original":1609728,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1081610240,"data_stored":1083259552,"data_compressed":11134625,"data_compressed_allocated":11599872,"data_compressed_original":13860864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1389703168,"data_stored":1392585856,"data_compressed":17106045,"data_compressed_allocated":17801216,"data_compressed_original":21614592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":953073664,"data_stored":957573512,"data_compressed":10069671,"data_compressed_allocated":10522624,"data_compressed_original":15568896,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":343392256,"data_stored":344816540,"data_compressed":5573327,"data_compressed_allocated":5816320,"data_compressed_original":7417856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1040125952,"data_stored":1041326796,"data_compressed":9059415,"data_compressed_allocated":9433088,"data_compressed_original":11280384,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T13:19:04.987 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T13:19:05.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.085+0000 7f5a151df640 1 Processor -- start 2024-08-19T13:19:05.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.085+0000 7f5a151df640 1 -- start start 2024-08-19T13:19:05.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a151df640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a1006f2e0 0x7f5a10072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a151df640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f5a1006f6c0 0x7f5a101432b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a151df640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a10072780 con 0x7f5a101437f0 2024-08-19T13:19:05.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a151df640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f5a1006ebe0 con 0x7f5a1006f6c0 2024-08-19T13:19:05.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a151df640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1006ee30 con 0x7f5a1006f2e0 2024-08-19T13:19:05.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a0ed9d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a1006f2e0 0x7f5a10072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a0659c640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f5a1006f6c0 0x7f5a101432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a0ed9d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a1006f2e0 0x7f5a10072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:50764/0 (socket says 172.21.15.103:50764) 2024-08-19T13:19:05.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a0659c640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f5a1006f6c0 0x7f5a101432b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47438/0 (socket says 172.21.15.103:47438) 2024-08-19T13:19:05.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a0ed9d640 1 -- 172.21.15.103:0/53552267 learned_addr learned my addr 172.21.15.103:0/53552267 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:05.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a0659c640 1 -- 172.21.15.103:0/53552267 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a1006f2e0 msgr2=0x7f5a10072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:05.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a0659c640 1 --2- 172.21.15.103:0/53552267 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a1006f2e0 0x7f5a10072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a0659c640 1 -- 172.21.15.103:0/53552267 >> v1:172.21.15.103:6789/0 conn(0x7f5a101437f0 legacy=0x7f5a1013c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a0659c640 1 -- 172.21.15.103:0/53552267 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a100ada70 con 0x7f5a1006f6c0 2024-08-19T13:19:05.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.086+0000 7f5a0ed9d640 1 --2- 172.21.15.103:0/53552267 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a1006f2e0 0x7f5a10072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:05.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.087+0000 7f5a0659c640 1 --2- 172.21.15.103:0/53552267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a1006f6c0 0x7f5a101432b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f59f80097a0 tx=0x7f59f802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856e02d85d43aed8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:05.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.087+0000 7f5a0e59c640 1 -- 172.21.15.103:0/53552267 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8037070 con 0x7f5a1006f6c0 2024-08-19T13:19:05.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.087+0000 7f5a0e59c640 1 -- 172.21.15.103:0/53552267 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59f802ae50 con 0x7f5a1006f6c0 2024-08-19T13:19:05.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.087+0000 7f5a0e59c640 1 -- 172.21.15.103:0/53552267 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8032660 con 0x7f5a1006f6c0 2024-08-19T13:19:05.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.088+0000 7f5a151df640 1 -- 172.21.15.103:0/53552267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a1006f6c0 msgr2=0x7f5a101432b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.088+0000 7f5a151df640 1 --2- 172.21.15.103:0/53552267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a1006f6c0 0x7f5a101432b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f59f80097a0 tx=0x7f59f802a100 comp rx=0 tx=0).stop 2024-08-19T13:19:05.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.088+0000 7f5a151df640 1 -- 172.21.15.103:0/53552267 shutdown_connections 2024-08-19T13:19:05.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.088+0000 7f5a151df640 1 --2- 172.21.15.103:0/53552267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a1006f6c0 0x7f5a101432b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.088+0000 7f5a151df640 1 --2- 172.21.15.103:0/53552267 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a1006f2e0 0x7f5a10072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.088+0000 7f5a151df640 1 -- 172.21.15.103:0/53552267 >> 172.21.15.103:0/53552267 conn(0x7f5a100b3430 msgr2=0x7f5a100b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:05.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.088+0000 7f5a151df640 1 -- 172.21.15.103:0/53552267 shutdown_connections 2024-08-19T13:19:05.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.088+0000 7f5a151df640 1 -- 172.21.15.103:0/53552267 wait complete. 2024-08-19T13:19:05.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.089+0000 7f5a151df640 1 Processor -- start 2024-08-19T13:19:05.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.089+0000 7f5a151df640 1 -- start start 2024-08-19T13:19:05.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a151df640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a1006f2e0 0x7f5a1013fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a151df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a1006f6c0 0x7f5a101460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a151df640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a101437f0 0x7f5a10146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a151df640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1013cc50 con 0x7f5a1006f6c0 2024-08-19T13:19:05.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a151df640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5a100b4540 con 0x7f5a1006f2e0 2024-08-19T13:19:05.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a151df640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1013cdd0 con 0x7f5a101437f0 2024-08-19T13:19:05.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a0659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a1006f6c0 0x7f5a101460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a0f59e640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a101437f0 0x7f5a10146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a0f59e640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a101437f0 0x7f5a10146600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:50780/0 (socket says 172.21.15.103:50780) 2024-08-19T13:19:05.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a0f59e640 1 -- 172.21.15.103:0/440672504 learned_addr learned my addr 172.21.15.103:0/440672504 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:05.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a0ed9d640 1 --2- 172.21.15.103:0/440672504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a1006f2e0 0x7f5a1013fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a0f59e640 1 -- 172.21.15.103:0/440672504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a1006f2e0 msgr2=0x7f5a1013fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a0f59e640 1 --2- 172.21.15.103:0/440672504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a1006f2e0 0x7f5a1013fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a0f59e640 1 -- 172.21.15.103:0/440672504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a1006f6c0 msgr2=0x7f5a101460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a0f59e640 1 --2- 172.21.15.103:0/440672504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a1006f6c0 0x7f5a101460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a0f59e640 1 -- 172.21.15.103:0/440672504 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1013ff50 con 0x7f5a101437f0 2024-08-19T13:19:05.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.090+0000 7f5a0659c640 1 --2- 172.21.15.103:0/440672504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a1006f6c0 0x7f5a101460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:05.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.091+0000 7f5a0f59e640 1 --2- 172.21.15.103:0/440672504 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a101437f0 0x7f5a10146600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800c680 tx=0x7f5a0800cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:05.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.091+0000 7f5a07fff640 1 -- 172.21.15.103:0/440672504 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08003b70 con 0x7f5a101437f0 2024-08-19T13:19:05.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.091+0000 7f5a07fff640 1 -- 172.21.15.103:0/440672504 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a08003d10 con 0x7f5a101437f0 2024-08-19T13:19:05.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.091+0000 7f5a151df640 1 -- 172.21.15.103:0/440672504 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a101363b0 con 0x7f5a101437f0 2024-08-19T13:19:05.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.091+0000 7f5a07fff640 1 -- 172.21.15.103:0/440672504 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08005310 con 0x7f5a101437f0 2024-08-19T13:19:05.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.091+0000 7f5a151df640 1 -- 172.21.15.103:0/440672504 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a10146d20 con 0x7f5a101437f0 2024-08-19T13:19:05.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.092+0000 7f5a151df640 1 -- 172.21.15.103:0/440672504 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a101449f0 con 0x7f5a101437f0 2024-08-19T13:19:05.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.092+0000 7f5a07fff640 1 -- 172.21.15.103:0/440672504 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a080054b0 con 0x7f5a101437f0 2024-08-19T13:19:05.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.093+0000 7f5a07fff640 1 --2- 172.21.15.103:0/440672504 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f59f006be30 0x7f59f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.093+0000 7f5a07fff640 1 -- 172.21.15.103:0/440672504 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7f5a0808ac60 con 0x7f5a101437f0 2024-08-19T13:19:05.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.093+0000 7f5a0ed9d640 1 --2- 172.21.15.103:0/440672504 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f59f006be30 0x7f59f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.093+0000 7f5a0ed9d640 1 --2- 172.21.15.103:0/440672504 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f59f006be30 0x7f59f006e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009b70 tx=0x7f5a00008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:05.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.097+0000 7f5a07fff640 1 -- 172.21.15.103:0/440672504 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f5a101449f0 con 0x7f5a101437f0 2024-08-19T13:19:05.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.262+0000 7f5a151df640 1 -- 172.21.15.103:0/440672504 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a1013b060 con 0x7f59f006be30 2024-08-19T13:19:05.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.265+0000 7f5a07fff640 1 -- 172.21.15.103:0/440672504 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117854 (secure 0 0 0) 0x7f5a1013b060 con 0x7f59f006be30 2024-08-19T13:19:05.267 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:19:05.267 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-08-19T13:19:05.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.267+0000 7f5a151df640 1 -- 172.21.15.103:0/440672504 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f59f006be30 msgr2=0x7f59f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.267+0000 7f5a151df640 1 --2- 172.21.15.103:0/440672504 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f59f006be30 0x7f59f006e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009b70 tx=0x7f5a00008040 comp rx=0 tx=0).stop 2024-08-19T13:19:05.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.267+0000 7f5a151df640 1 -- 172.21.15.103:0/440672504 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a101437f0 msgr2=0x7f5a10146600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.267+0000 7f5a151df640 1 --2- 172.21.15.103:0/440672504 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a101437f0 0x7f5a10146600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800c680 tx=0x7f5a0800cb40 comp rx=0 tx=0).stop 2024-08-19T13:19:05.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.268+0000 7f5a151df640 1 -- 172.21.15.103:0/440672504 shutdown_connections 2024-08-19T13:19:05.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.268+0000 7f5a151df640 1 --2- 172.21.15.103:0/440672504 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f59f006be30 0x7f59f006e2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.268+0000 7f5a151df640 1 --2- 172.21.15.103:0/440672504 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a101437f0 0x7f5a10146600 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.268+0000 7f5a151df640 1 --2- 172.21.15.103:0/440672504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a1006f6c0 0x7f5a101460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.268+0000 7f5a151df640 1 --2- 172.21.15.103:0/440672504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a1006f2e0 0x7f5a1013fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.268+0000 7f5a151df640 1 -- 172.21.15.103:0/440672504 >> 172.21.15.103:0/440672504 conn(0x7f5a100b3430 msgr2=0x7f5a100733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:05.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.268+0000 7f5a151df640 1 -- 172.21.15.103:0/440672504 shutdown_connections 2024-08-19T13:19:05.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.268+0000 7f5a151df640 1 -- 172.21.15.103:0/440672504 wait complete. 2024-08-19T13:19:05.279 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-08-19T13:19:04.047415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3575960891,"num_objects":884,"num_object_clones":0,"num_object_copies":1768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":884,"num_whiteouts":0,"num_read":2623,"num_read_kb":2618,"num_write":162624,"num_write_kb":5650011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":165414,"ondisk_log_size":165414,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9309976,"kb_used_data":6675992,"kb_used_omap":12,"kb_used_meta":2633907,"kb_avail":745664744,"statfs":{"total":773094113280,"available":763560697856,"internally_reserved":0,"allocated":6836215808,"data_stored":6851292878,"data_compressed":70441838,"data_compressed_allocated":73416704,"data_compressed_original":94011392,"omap_allocated":12752,"internal_metadata":2697121328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40960,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002825"},"pg_stats":[{"pgid":"3.1f","version":"22'3551","reported_seq":3577,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.759901+0000","last_change":"2024-08-19T13:15:04.704168+0000","last_active":"2024-08-19T13:17:47.759901+0000","last_peered":"2024-08-19T13:17:47.759901+0000","last_clean":"2024-08-19T13:17:47.759901+0000","last_became_active":"2024-08-19T13:15:04.703897+0000","last_became_peered":"2024-08-19T13:15:04.703897+0000","last_unstale":"2024-08-19T13:17:47.759901+0000","last_undegraded":"2024-08-19T13:17:47.759901+0000","last_fullsized":"2024-08-19T13:17:47.759901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3551,"log_dups_size":0,"ondisk_log_size":3551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:54:14.098742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":3566,"num_write_kb":193154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'5748","reported_seq":5766,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:54.365605+0000","last_change":"2024-08-19T13:15:04.702034+0000","last_active":"2024-08-19T13:17:54.365605+0000","last_peered":"2024-08-19T13:17:54.365605+0000","last_clean":"2024-08-19T13:17:54.365605+0000","last_became_active":"2024-08-19T13:15:04.701653+0000","last_became_peered":"2024-08-19T13:15:04.701653+0000","last_unstale":"2024-08-19T13:17:54.365605+0000","last_undegraded":"2024-08-19T13:17:54.365605+0000","last_fullsized":"2024-08-19T13:17:54.365605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5748,"log_dups_size":0,"ondisk_log_size":5748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T15:23:44.488137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128583504,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5733,"num_write_kb":196004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"23'5874","reported_seq":5890,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:00.011644+0000","last_change":"2024-08-19T13:15:04.708047+0000","last_active":"2024-08-19T13:18:00.011644+0000","last_peered":"2024-08-19T13:18:00.011644+0000","last_clean":"2024-08-19T13:18:00.011644+0000","last_became_active":"2024-08-19T13:15:04.707756+0000","last_became_peered":"2024-08-19T13:15:04.707756+0000","last_unstale":"2024-08-19T13:18:00.011644+0000","last_undegraded":"2024-08-19T13:18:00.011644+0000","last_fullsized":"2024-08-19T13:18:00.011644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5874,"log_dups_size":0,"ondisk_log_size":5874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:23:19.211747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113312264,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5859,"num_write_kb":177354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"22'1998","reported_seq":2013,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.729767+0000","last_change":"2024-08-19T13:15:04.707039+0000","last_active":"2024-08-19T13:17:47.729767+0000","last_peered":"2024-08-19T13:17:47.729767+0000","last_clean":"2024-08-19T13:17:47.729767+0000","last_became_active":"2024-08-19T13:15:04.706726+0000","last_became_peered":"2024-08-19T13:15:04.706726+0000","last_unstale":"2024-08-19T13:17:47.729767+0000","last_undegraded":"2024-08-19T13:17:47.729767+0000","last_fullsized":"2024-08-19T13:17:47.729767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":1998,"log_dups_size":0,"ondisk_log_size":1998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:45:09.075439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1990,"num_write_kb":94721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"23'3781","reported_seq":3797,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.969389+0000","last_change":"2024-08-19T13:15:04.707093+0000","last_active":"2024-08-19T13:17:47.969389+0000","last_peered":"2024-08-19T13:17:47.969389+0000","last_clean":"2024-08-19T13:17:47.969389+0000","last_became_active":"2024-08-19T13:15:04.706832+0000","last_became_peered":"2024-08-19T13:15:04.706832+0000","last_unstale":"2024-08-19T13:17:47.969389+0000","last_undegraded":"2024-08-19T13:17:47.969389+0000","last_fullsized":"2024-08-19T13:17:47.969389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3781,"log_dups_size":0,"ondisk_log_size":3781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T15:22:00.995458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104858112,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3741,"num_write_kb":132744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"23'7047","reported_seq":7066,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:53.256999+0000","last_change":"2024-08-19T13:15:04.706943+0000","last_active":"2024-08-19T13:17:53.256999+0000","last_peered":"2024-08-19T13:17:53.256999+0000","last_clean":"2024-08-19T13:17:53.256999+0000","last_became_active":"2024-08-19T13:15:04.706371+0000","last_became_peered":"2024-08-19T13:15:04.706371+0000","last_unstale":"2024-08-19T13:17:53.256999+0000","last_undegraded":"2024-08-19T13:17:53.256999+0000","last_fullsized":"2024-08-19T13:17:53.256999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":7047,"log_dups_size":0,"ondisk_log_size":7047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:36:21.721116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134217728,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":7029,"num_write_kb":256635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"23'4609","reported_seq":4626,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:04.184395+0000","last_change":"2024-08-19T13:15:04.701266+0000","last_active":"2024-08-19T13:18:04.184395+0000","last_peered":"2024-08-19T13:18:04.184395+0000","last_clean":"2024-08-19T13:18:04.184395+0000","last_became_active":"2024-08-19T13:15:04.700993+0000","last_became_peered":"2024-08-19T13:15:04.700993+0000","last_unstale":"2024-08-19T13:18:04.184395+0000","last_undegraded":"2024-08-19T13:18:04.184395+0000","last_fullsized":"2024-08-19T13:18:04.184395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4609,"log_dups_size":0,"ondisk_log_size":4609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:01:09.023570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109052416,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":4571,"num_write_kb":180994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"23'6372","reported_seq":6387,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:04.058559+0000","last_change":"2024-08-19T13:15:04.699913+0000","last_active":"2024-08-19T13:18:04.058559+0000","last_peered":"2024-08-19T13:18:04.058559+0000","last_clean":"2024-08-19T13:18:04.058559+0000","last_became_active":"2024-08-19T13:15:04.699219+0000","last_became_peered":"2024-08-19T13:15:04.699219+0000","last_unstale":"2024-08-19T13:18:04.058559+0000","last_undegraded":"2024-08-19T13:18:04.058559+0000","last_fullsized":"2024-08-19T13:18:04.058559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":6372,"log_dups_size":0,"ondisk_log_size":6372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:00:52.367696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6366,"num_write_kb":136778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"23'7655","reported_seq":7673,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:07.987425+0000","last_change":"2024-08-19T13:15:04.701157+0000","last_active":"2024-08-19T13:18:07.987425+0000","last_peered":"2024-08-19T13:18:07.987425+0000","last_clean":"2024-08-19T13:18:07.987425+0000","last_became_active":"2024-08-19T13:15:04.700904+0000","last_became_peered":"2024-08-19T13:15:04.700904+0000","last_unstale":"2024-08-19T13:18:07.987425+0000","last_undegraded":"2024-08-19T13:18:07.987425+0000","last_fullsized":"2024-08-19T13:18:07.987425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":7655,"log_dups_size":0,"ondisk_log_size":7655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:41:45.337705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146932240,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":7658,"num_write_kb":230670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"23'3889","reported_seq":3904,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.917748+0000","last_change":"2024-08-19T13:15:04.704857+0000","last_active":"2024-08-19T13:17:47.917748+0000","last_peered":"2024-08-19T13:17:47.917748+0000","last_clean":"2024-08-19T13:17:47.917748+0000","last_became_active":"2024-08-19T13:15:04.704383+0000","last_became_peered":"2024-08-19T13:15:04.704383+0000","last_unstale":"2024-08-19T13:17:47.917748+0000","last_undegraded":"2024-08-19T13:17:47.917748+0000","last_fullsized":"2024-08-19T13:17:47.917748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3889,"log_dups_size":0,"ondisk_log_size":3889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:27:48.271684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106366632,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3864,"num_write_kb":162549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"23'8294","reported_seq":8332,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:03.337243+0000","last_change":"2024-08-19T13:15:04.703573+0000","last_active":"2024-08-19T13:18:03.337243+0000","last_peered":"2024-08-19T13:18:03.337243+0000","last_clean":"2024-08-19T13:18:03.337243+0000","last_became_active":"2024-08-19T13:15:04.703277+0000","last_became_peered":"2024-08-19T13:15:04.703277+0000","last_unstale":"2024-08-19T13:18:03.337243+0000","last_undegraded":"2024-08-19T13:18:03.337243+0000","last_fullsized":"2024-08-19T13:18:03.337243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":8294,"log_dups_size":0,"ondisk_log_size":8294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:08:33.844086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125960192,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":85,"num_read_kb":132,"num_write":8326,"num_write_kb":182642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"23'6457","reported_seq":6573,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:57.290424+0000","last_change":"2024-08-19T13:15:04.707221+0000","last_active":"2024-08-19T13:17:57.290424+0000","last_peered":"2024-08-19T13:17:57.290424+0000","last_clean":"2024-08-19T13:17:57.290424+0000","last_became_active":"2024-08-19T13:15:04.706496+0000","last_became_peered":"2024-08-19T13:15:04.706496+0000","last_unstale":"2024-08-19T13:17:57.290424+0000","last_undegraded":"2024-08-19T13:17:57.290424+0000","last_fullsized":"2024-08-19T13:17:57.290424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":6457,"log_dups_size":0,"ondisk_log_size":6457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:43:34.377186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90964496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":6419,"num_write_kb":175583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.730287+0000","last_change":"2024-08-19T13:14:58.168503+0000","last_active":"2024-08-19T13:17:47.730287+0000","last_peered":"2024-08-19T13:17:47.730287+0000","last_clean":"2024-08-19T13:17:47.730287+0000","last_became_active":"2024-08-19T13:14:56.044662+0000","last_became_peered":"2024-08-19T13:14:56.044662+0000","last_unstale":"2024-08-19T13:17:47.730287+0000","last_undegraded":"2024-08-19T13:17:47.730287+0000","last_fullsized":"2024-08-19T13:17:47.730287+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T23:26:23.570412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039371300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'6792","reported_seq":6816,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:09.561558+0000","last_change":"2024-08-19T13:15:04.700796+0000","last_active":"2024-08-19T13:18:09.561558+0000","last_peered":"2024-08-19T13:18:09.561558+0000","last_clean":"2024-08-19T13:18:09.561558+0000","last_became_active":"2024-08-19T13:15:04.700540+0000","last_became_peered":"2024-08-19T13:15:04.700540+0000","last_unstale":"2024-08-19T13:18:09.561558+0000","last_undegraded":"2024-08-19T13:18:09.561558+0000","last_fullsized":"2024-08-19T13:18:09.561558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":6792,"log_dups_size":0,"ondisk_log_size":6792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:39:03.153240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104988672,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":34,"num_read_kb":151,"num_write":6787,"num_write_kb":141259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.758067+0000","last_change":"2024-08-19T13:14:58.061578+0000","last_active":"2024-08-19T13:17:47.758067+0000","last_peered":"2024-08-19T13:17:47.758067+0000","last_clean":"2024-08-19T13:17:47.758067+0000","last_became_active":"2024-08-19T13:14:56.045731+0000","last_became_peered":"2024-08-19T13:14:56.045731+0000","last_unstale":"2024-08-19T13:17:47.758067+0000","last_undegraded":"2024-08-19T13:17:47.758067+0000","last_fullsized":"2024-08-19T13:17:47.758067+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:49:02.156480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044000000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'5379","reported_seq":5402,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.968669+0000","last_change":"2024-08-19T13:15:04.703333+0000","last_active":"2024-08-19T13:17:47.968669+0000","last_peered":"2024-08-19T13:17:47.968669+0000","last_clean":"2024-08-19T13:17:47.968669+0000","last_became_active":"2024-08-19T13:15:04.703011+0000","last_became_peered":"2024-08-19T13:15:04.703011+0000","last_unstale":"2024-08-19T13:17:47.968669+0000","last_undegraded":"2024-08-19T13:17:47.968669+0000","last_fullsized":"2024-08-19T13:17:47.968669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5379,"log_dups_size":0,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:08:52.537445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146276352,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5343,"num_write_kb":213509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.740138+0000","last_change":"2024-08-19T13:14:58.373877+0000","last_active":"2024-08-19T13:17:47.740138+0000","last_peered":"2024-08-19T13:17:47.740138+0000","last_clean":"2024-08-19T13:17:47.740138+0000","last_became_active":"2024-08-19T13:14:56.111427+0000","last_became_peered":"2024-08-19T13:14:56.111427+0000","last_unstale":"2024-08-19T13:17:47.740138+0000","last_undegraded":"2024-08-19T13:17:47.740138+0000","last_fullsized":"2024-08-19T13:17:47.740138+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:19:29.396593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050374000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'8281","reported_seq":8296,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:06.692281+0000","last_change":"2024-08-19T13:15:04.699221+0000","last_active":"2024-08-19T13:18:06.692281+0000","last_peered":"2024-08-19T13:18:06.692281+0000","last_clean":"2024-08-19T13:18:06.692281+0000","last_became_active":"2024-08-19T13:15:04.698944+0000","last_became_peered":"2024-08-19T13:15:04.698944+0000","last_unstale":"2024-08-19T13:18:06.692281+0000","last_undegraded":"2024-08-19T13:18:06.692281+0000","last_fullsized":"2024-08-19T13:18:06.692281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":8281,"log_dups_size":0,"ondisk_log_size":8281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:45:55.451611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8278,"num_write_kb":279553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.757967+0000","last_change":"2024-08-19T13:14:58.061576+0000","last_active":"2024-08-19T13:17:47.757967+0000","last_peered":"2024-08-19T13:17:47.757967+0000","last_clean":"2024-08-19T13:17:47.757967+0000","last_became_active":"2024-08-19T13:14:56.111065+0000","last_became_peered":"2024-08-19T13:14:56.111065+0000","last_unstale":"2024-08-19T13:17:47.757967+0000","last_undegraded":"2024-08-19T13:17:47.757967+0000","last_fullsized":"2024-08-19T13:17:47.757967+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:43:13.276725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042019699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'4283","reported_seq":4312,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:52.878133+0000","last_change":"2024-08-19T13:15:04.698014+0000","last_active":"2024-08-19T13:18:52.878133+0000","last_peered":"2024-08-19T13:18:52.878133+0000","last_clean":"2024-08-19T13:18:52.878133+0000","last_became_active":"2024-08-19T13:15:04.697675+0000","last_became_peered":"2024-08-19T13:15:04.697675+0000","last_unstale":"2024-08-19T13:18:52.878133+0000","last_undegraded":"2024-08-19T13:18:52.878133+0000","last_fullsized":"2024-08-19T13:18:52.878133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4283,"log_dups_size":0,"ondisk_log_size":4283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:28:29.123627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114253824,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":58,"num_read_kb":45,"num_write":4306,"num_write_kb":159578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'5377","reported_seq":5407,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.733133+0000","last_change":"2024-08-19T13:15:04.703723+0000","last_active":"2024-08-19T13:17:47.733133+0000","last_peered":"2024-08-19T13:17:47.733133+0000","last_clean":"2024-08-19T13:17:47.733133+0000","last_became_active":"2024-08-19T13:15:04.703417+0000","last_became_peered":"2024-08-19T13:15:04.703417+0000","last_unstale":"2024-08-19T13:17:47.733133+0000","last_undegraded":"2024-08-19T13:17:47.733133+0000","last_fullsized":"2024-08-19T13:17:47.733133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5377,"log_dups_size":0,"ondisk_log_size":5377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:11:03.088099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":5405,"num_write_kb":165854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.729507+0000","last_change":"2024-08-19T13:14:58.056552+0000","last_active":"2024-08-19T13:17:47.729507+0000","last_peered":"2024-08-19T13:17:47.729507+0000","last_clean":"2024-08-19T13:17:47.729507+0000","last_became_active":"2024-08-19T13:14:56.045753+0000","last_became_peered":"2024-08-19T13:14:56.045753+0000","last_unstale":"2024-08-19T13:17:47.729507+0000","last_undegraded":"2024-08-19T13:17:47.729507+0000","last_fullsized":"2024-08-19T13:17:47.729507+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:12:21.860038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037158599999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"23'2823","reported_seq":2843,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.965052+0000","last_change":"2024-08-19T13:15:04.699545+0000","last_active":"2024-08-19T13:17:47.965052+0000","last_peered":"2024-08-19T13:17:47.965052+0000","last_clean":"2024-08-19T13:17:47.965052+0000","last_became_active":"2024-08-19T13:15:04.699255+0000","last_became_peered":"2024-08-19T13:15:04.699255+0000","last_unstale":"2024-08-19T13:17:47.965052+0000","last_undegraded":"2024-08-19T13:17:47.965052+0000","last_fullsized":"2024-08-19T13:17:47.965052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":2823,"log_dups_size":0,"ondisk_log_size":2823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:46:06.725740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84017152,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":2784,"num_write_kb":135939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.736581+0000","last_change":"2024-08-19T13:14:58.055759+0000","last_active":"2024-08-19T13:17:47.736581+0000","last_peered":"2024-08-19T13:17:47.736581+0000","last_clean":"2024-08-19T13:17:47.736581+0000","last_became_active":"2024-08-19T13:14:56.046524+0000","last_became_peered":"2024-08-19T13:14:56.046524+0000","last_unstale":"2024-08-19T13:17:47.736581+0000","last_undegraded":"2024-08-19T13:17:47.736581+0000","last_fullsized":"2024-08-19T13:17:47.736581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T23:05:11.782761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043587900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"23'5930","reported_seq":5946,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.915146+0000","last_change":"2024-08-19T13:15:04.710024+0000","last_active":"2024-08-19T13:17:47.915146+0000","last_peered":"2024-08-19T13:17:47.915146+0000","last_clean":"2024-08-19T13:17:47.915146+0000","last_became_active":"2024-08-19T13:15:04.709768+0000","last_became_peered":"2024-08-19T13:15:04.709768+0000","last_unstale":"2024-08-19T13:17:47.915146+0000","last_undegraded":"2024-08-19T13:17:47.915146+0000","last_fullsized":"2024-08-19T13:17:47.915146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5930,"log_dups_size":0,"ondisk_log_size":5930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:26:22.787995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146866696,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5915,"num_write_kb":254104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.739748+0000","last_change":"2024-08-19T13:14:58.373682+0000","last_active":"2024-08-19T13:17:47.739748+0000","last_peered":"2024-08-19T13:17:47.739748+0000","last_clean":"2024-08-19T13:17:47.739748+0000","last_became_active":"2024-08-19T13:14:56.045391+0000","last_became_peered":"2024-08-19T13:14:56.045391+0000","last_unstale":"2024-08-19T13:17:47.739748+0000","last_undegraded":"2024-08-19T13:17:47.739748+0000","last_fullsized":"2024-08-19T13:17:47.739748+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:41:33.266324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037776800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"23'5375","reported_seq":5407,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:01.600150+0000","last_change":"2024-08-19T13:15:04.698787+0000","last_active":"2024-08-19T13:18:01.600150+0000","last_peered":"2024-08-19T13:18:01.600150+0000","last_clean":"2024-08-19T13:18:01.600150+0000","last_became_active":"2024-08-19T13:15:04.698015+0000","last_became_peered":"2024-08-19T13:15:04.698015+0000","last_unstale":"2024-08-19T13:18:01.600150+0000","last_undegraded":"2024-08-19T13:18:01.600150+0000","last_fullsized":"2024-08-19T13:18:01.600150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5375,"log_dups_size":0,"ondisk_log_size":5375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:07:36.161135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":68,"num_read_kb":114,"num_write":5405,"num_write_kb":156434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.729454+0000","last_change":"2024-08-19T13:14:58.056578+0000","last_active":"2024-08-19T13:17:47.729454+0000","last_peered":"2024-08-19T13:17:47.729454+0000","last_clean":"2024-08-19T13:17:47.729454+0000","last_became_active":"2024-08-19T13:14:56.045464+0000","last_became_peered":"2024-08-19T13:14:56.045464+0000","last_unstale":"2024-08-19T13:17:47.729454+0000","last_undegraded":"2024-08-19T13:17:47.729454+0000","last_fullsized":"2024-08-19T13:17:47.729454+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:35:16.028455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039910299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.739769+0000","last_change":"2024-08-19T13:14:55.043964+0000","last_active":"2024-08-19T13:17:47.739769+0000","last_peered":"2024-08-19T13:17:47.739769+0000","last_clean":"2024-08-19T13:17:47.739769+0000","last_became_active":"2024-08-19T13:14:55.043626+0000","last_became_peered":"2024-08-19T13:14:55.043626+0000","last_unstale":"2024-08-19T13:17:47.739769+0000","last_undegraded":"2024-08-19T13:17:47.739769+0000","last_fullsized":"2024-08-19T13:17:47.739769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:54.027963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:54.027963+0000","last_clean_scrub_stamp":"2024-08-19T13:14:54.027963+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:36:05.876096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"23'2767","reported_seq":2782,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.965472+0000","last_change":"2024-08-19T13:15:04.699203+0000","last_active":"2024-08-19T13:17:47.965472+0000","last_peered":"2024-08-19T13:17:47.965472+0000","last_clean":"2024-08-19T13:17:47.965472+0000","last_became_active":"2024-08-19T13:15:04.698903+0000","last_became_peered":"2024-08-19T13:15:04.698903+0000","last_unstale":"2024-08-19T13:17:47.965472+0000","last_undegraded":"2024-08-19T13:17:47.965472+0000","last_fullsized":"2024-08-19T13:17:47.965472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:32:06.151208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2699,"num_write_kb":115587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"23'9933","reported_seq":10062,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:04.629618+0000","last_change":"2024-08-19T13:15:04.708930+0000","last_active":"2024-08-19T13:18:04.629618+0000","last_peered":"2024-08-19T13:18:04.629618+0000","last_clean":"2024-08-19T13:18:04.629618+0000","last_became_active":"2024-08-19T13:15:04.706442+0000","last_became_peered":"2024-08-19T13:15:04.706442+0000","last_unstale":"2024-08-19T13:18:04.629618+0000","last_undegraded":"2024-08-19T13:18:04.629618+0000","last_fullsized":"2024-08-19T13:18:04.629618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":9933,"log_dups_size":0,"ondisk_log_size":9933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:18:30.218698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145424384,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":361,"num_read_kb":322,"num_write":6867,"num_write_kb":212554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"23'5336","reported_seq":5354,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:05.747745+0000","last_change":"2024-08-19T13:15:04.708013+0000","last_active":"2024-08-19T13:18:05.747745+0000","last_peered":"2024-08-19T13:18:05.747745+0000","last_clean":"2024-08-19T13:18:05.747745+0000","last_became_active":"2024-08-19T13:15:04.707724+0000","last_became_peered":"2024-08-19T13:15:04.707724+0000","last_unstale":"2024-08-19T13:18:05.747745+0000","last_undegraded":"2024-08-19T13:18:05.747745+0000","last_fullsized":"2024-08-19T13:18:05.747745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:59:28.795104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":11,"num_read_kb":71,"num_write":5419,"num_write_kb":161478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"22'3547","reported_seq":3562,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.729697+0000","last_change":"2024-08-19T13:15:04.700346+0000","last_active":"2024-08-19T13:17:47.729697+0000","last_peered":"2024-08-19T13:17:47.729697+0000","last_clean":"2024-08-19T13:17:47.729697+0000","last_became_active":"2024-08-19T13:15:04.699228+0000","last_became_peered":"2024-08-19T13:15:04.699228+0000","last_unstale":"2024-08-19T13:17:47.729697+0000","last_undegraded":"2024-08-19T13:17:47.729697+0000","last_fullsized":"2024-08-19T13:17:47.729697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3547,"log_dups_size":0,"ondisk_log_size":3547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T23:29:48.186111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3536,"num_write_kb":144645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"23'3397","reported_seq":3441,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.967033+0000","last_change":"2024-08-19T13:15:04.699676+0000","last_active":"2024-08-19T13:17:47.967033+0000","last_peered":"2024-08-19T13:17:47.967033+0000","last_clean":"2024-08-19T13:17:47.967033+0000","last_became_active":"2024-08-19T13:15:04.699383+0000","last_became_peered":"2024-08-19T13:15:04.699383+0000","last_unstale":"2024-08-19T13:17:47.967033+0000","last_undegraded":"2024-08-19T13:17:47.967033+0000","last_fullsized":"2024-08-19T13:17:47.967033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3397,"log_dups_size":0,"ondisk_log_size":3397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:10:11.285767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":121,"num_read_kb":99,"num_write":3411,"num_write_kb":138057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'4376","reported_seq":4398,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.916399+0000","last_change":"2024-08-19T13:15:04.709245+0000","last_active":"2024-08-19T13:17:47.916399+0000","last_peered":"2024-08-19T13:17:47.916399+0000","last_clean":"2024-08-19T13:17:47.916399+0000","last_became_active":"2024-08-19T13:15:04.708174+0000","last_became_peered":"2024-08-19T13:15:04.708174+0000","last_unstale":"2024-08-19T13:17:47.916399+0000","last_undegraded":"2024-08-19T13:17:47.916399+0000","last_fullsized":"2024-08-19T13:17:47.916399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4376,"log_dups_size":0,"ondisk_log_size":4376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:25:57.887618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123076608,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":27,"num_read_kb":82,"num_write":4364,"num_write_kb":209707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'4908","reported_seq":5001,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:48.004070+0000","last_change":"2024-08-19T13:15:04.707739+0000","last_active":"2024-08-19T13:17:48.004070+0000","last_peered":"2024-08-19T13:17:48.004070+0000","last_clean":"2024-08-19T13:17:48.004070+0000","last_became_active":"2024-08-19T13:15:04.707482+0000","last_became_peered":"2024-08-19T13:15:04.707482+0000","last_unstale":"2024-08-19T13:17:48.004070+0000","last_undegraded":"2024-08-19T13:17:48.004070+0000","last_fullsized":"2024-08-19T13:17:48.004070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4908,"log_dups_size":0,"ondisk_log_size":4908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:07:38.674944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123349040,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":288,"num_read_kb":216,"num_write":5188,"num_write_kb":204415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'3580","reported_seq":3595,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.969742+0000","last_change":"2024-08-19T13:15:04.707186+0000","last_active":"2024-08-19T13:17:47.969742+0000","last_peered":"2024-08-19T13:17:47.969742+0000","last_clean":"2024-08-19T13:17:47.969742+0000","last_became_active":"2024-08-19T13:15:04.706891+0000","last_became_peered":"2024-08-19T13:15:04.706891+0000","last_unstale":"2024-08-19T13:17:47.969742+0000","last_undegraded":"2024-08-19T13:17:47.969742+0000","last_fullsized":"2024-08-19T13:17:47.969742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3580,"log_dups_size":0,"ondisk_log_size":3580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:45:10.115661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114560672,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3532,"num_write_kb":190257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"23'2893","reported_seq":2908,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.966115+0000","last_change":"2024-08-19T13:15:04.704399+0000","last_active":"2024-08-19T13:17:47.966115+0000","last_peered":"2024-08-19T13:17:47.966115+0000","last_clean":"2024-08-19T13:17:47.966115+0000","last_became_active":"2024-08-19T13:15:04.704113+0000","last_became_peered":"2024-08-19T13:15:04.704113+0000","last_unstale":"2024-08-19T13:17:47.966115+0000","last_undegraded":"2024-08-19T13:17:47.966115+0000","last_fullsized":"2024-08-19T13:17:47.966115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":2893,"log_dups_size":0,"ondisk_log_size":2893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:09:37.265840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103415808,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2845,"num_write_kb":157315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"22'4138","reported_seq":4158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.739708+0000","last_change":"2024-08-19T13:15:04.709668+0000","last_active":"2024-08-19T13:17:47.739708+0000","last_peered":"2024-08-19T13:17:47.739708+0000","last_clean":"2024-08-19T13:17:47.739708+0000","last_became_active":"2024-08-19T13:15:04.709376+0000","last_became_peered":"2024-08-19T13:15:04.709376+0000","last_unstale":"2024-08-19T13:17:47.739708+0000","last_undegraded":"2024-08-19T13:17:47.739708+0000","last_fullsized":"2024-08-19T13:17:47.739708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:26:41.219376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":4141,"num_write_kb":181515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"23'3049","reported_seq":3079,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.966674+0000","last_change":"2024-08-19T13:15:04.704226+0000","last_active":"2024-08-19T13:17:47.966674+0000","last_peered":"2024-08-19T13:17:47.966674+0000","last_clean":"2024-08-19T13:17:47.966674+0000","last_became_active":"2024-08-19T13:15:04.703965+0000","last_became_peered":"2024-08-19T13:15:04.703965+0000","last_unstale":"2024-08-19T13:17:47.966674+0000","last_undegraded":"2024-08-19T13:17:47.966674+0000","last_fullsized":"2024-08-19T13:17:47.966674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3049,"log_dups_size":0,"ondisk_log_size":3049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:51:52.289558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79700496,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":58,"num_read_kb":46,"num_write":3047,"num_write_kb":132115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"23'7855","reported_seq":8198,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:59.958424+0000","last_change":"2024-08-19T13:15:04.707304+0000","last_active":"2024-08-19T13:17:59.958424+0000","last_peered":"2024-08-19T13:17:59.958424+0000","last_clean":"2024-08-19T13:17:59.958424+0000","last_became_active":"2024-08-19T13:15:04.706952+0000","last_became_peered":"2024-08-19T13:15:04.706952+0000","last_unstale":"2024-08-19T13:17:59.958424+0000","last_undegraded":"2024-08-19T13:17:59.958424+0000","last_fullsized":"2024-08-19T13:17:59.958424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":7855,"log_dups_size":0,"ondisk_log_size":7855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:27:03.627133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1110,"num_read_kb":992,"num_write":8069,"num_write_kb":173890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3575370504,"num_objects":880,"num_object_clones":0,"num_object_copies":1760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":880,"num_whiteouts":0,"num_read":2507,"num_read_kb":2518,"num_write":162463,"num_write_kb":5647593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6834151424,"data_stored":6849530008,"data_compressed":70382846,"data_compressed_allocated":73334784,"data_compressed_original":92987392,"omap_allocated":0,"internal_metadata":0},"log_size":165294,"ondisk_log_size":165294,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":58992,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1448372,"kb_used_data":1016044,"kb_used_omap":1,"kb_used_meta":432318,"kb_avail":92923468,"statfs":{"total":96636764160,"available":95153631232,"internally_reserved":0,"allocated":1040429056,"data_stored":1041943846,"data_compressed":9088911,"data_compressed_allocated":9474048,"data_compressed_original":11792384,"omap_allocated":1590,"internal_metadata":442694090},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443108,"kb_used_data":335580,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":93928732,"statfs":{"total":96636764160,"available":96183021568,"internally_reserved":0,"allocated":343633920,"data_stored":344904662,"data_compressed":5573327,"data_compressed_allocated":5816320,"data_compressed_original":7417856,"omap_allocated":1594,"internal_metadata":110098886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1250856,"kb_used_data":930976,"kb_used_omap":1,"kb_used_meta":319870,"kb_avail":93120984,"statfs":{"total":96636764160,"available":95355887616,"internally_reserved":0,"allocated":953319424,"data_stored":957661653,"data_compressed":10069671,"data_compressed_allocated":10522624,"data_compressed_original":15568896,"omap_allocated":1595,"internal_metadata":327547333},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1865984,"kb_used_data":1357368,"kb_used_omap":1,"kb_used_meta":508606,"kb_avail":92505856,"statfs":{"total":96636764160,"available":94725996544,"internally_reserved":0,"allocated":1389944832,"data_stored":1392673978,"data_compressed":17106045,"data_compressed_allocated":17801216,"data_compressed_original":21614592,"omap_allocated":1595,"internal_metadata":520812997},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1480504,"kb_used_data":1056496,"kb_used_omap":1,"kb_used_meta":423998,"kb_avail":92891336,"statfs":{"total":96636764160,"available":95120728064,"internally_reserved":0,"allocated":1081851904,"data_stored":1083347674,"data_compressed":11134625,"data_compressed_allocated":11599872,"data_compressed_original":13860864,"omap_allocated":1593,"internal_metadata":434174407},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496100,"kb_used_data":338652,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":93875740,"statfs":{"total":96636764160,"available":96128757760,"internally_reserved":0,"allocated":346779648,"data_stored":346707202,"data_compressed":1288748,"data_compressed_allocated":1335296,"data_compressed_original":1609728,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":874888,"kb_used_data":600832,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":93496952,"statfs":{"total":96636764160,"available":95740878848,"internally_reserved":0,"allocated":615251968,"data_stored":615348941,"data_compressed":3731208,"data_compressed_allocated":3866624,"data_compressed_original":4513792,"omap_allocated":1595,"internal_metadata":280623557},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1450164,"kb_used_data":1040044,"kb_used_omap":1,"kb_used_meta":410110,"kb_avail":92921676,"statfs":{"total":96636764160,"available":95151796224,"internally_reserved":0,"allocated":1065005056,"data_stored":1068704922,"data_compressed":12449303,"data_compressed_allocated":13000704,"data_compressed_original":17633280,"omap_allocated":1595,"internal_metadata":419953093},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1064701952,"data_stored":1068087872,"data_compressed":12419807,"data_compressed_allocated":12959744,"data_compressed_original":17121280,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":615006208,"data_stored":615260800,"data_compressed":3731208,"data_compressed_allocated":3866624,"data_compressed_original":4513792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346537984,"data_stored":346619080,"data_compressed":1288748,"data_compressed_allocated":1335296,"data_compressed_original":1609728,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1081610240,"data_stored":1083259552,"data_compressed":11134625,"data_compressed_allocated":11599872,"data_compressed_original":13860864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1389703168,"data_stored":1392585856,"data_compressed":17106045,"data_compressed_allocated":17801216,"data_compressed_original":21614592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":953073664,"data_stored":957573512,"data_compressed":10069671,"data_compressed_allocated":10522624,"data_compressed_original":15568896,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":343392256,"data_stored":344816540,"data_compressed":5573327,"data_compressed_allocated":5816320,"data_compressed_original":7417856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1040125952,"data_stored":1041326796,"data_compressed":9059415,"data_compressed_allocated":9433088,"data_compressed_original":11280384,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T13:19:05.281 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-19T13:19:05.282 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T13:19:05.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.379+0000 7f9039df8640 1 Processor -- start 2024-08-19T13:19:05.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.380+0000 7f9039df8640 1 -- start start 2024-08-19T13:19:05.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.380+0000 7f9039df8640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f90341375a0 0x7f90340a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.380+0000 7f9039df8640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9034143f10 0x7f90341443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.380+0000 7f9039df8640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f903406cbc0 con 0x7f903406c7e0 2024-08-19T13:19:05.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.380+0000 7f9039df8640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f90340ab680 con 0x7f90341375a0 2024-08-19T13:19:05.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.380+0000 7f9039df8640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f90340ab960 con 0x7f9034143f10 2024-08-19T13:19:05.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.380+0000 7f9032ffd640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f90341375a0 0x7f90340a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.380+0000 7f9033fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9034143f10 0x7f90341443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.380+0000 7f9032ffd640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f90341375a0 0x7f90340a9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47460/0 (socket says 172.21.15.103:47460) 2024-08-19T13:19:05.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.380+0000 7f9033fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9034143f10 0x7f90341443c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:50794/0 (socket says 172.21.15.103:50794) 2024-08-19T13:19:05.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.380+0000 7f9032ffd640 1 -- 172.21.15.103:0/3145624712 learned_addr learned my addr 172.21.15.103:0/3145624712 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:05.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.381+0000 7f9033fff640 1 -- 172.21.15.103:0/3145624712 >> v2:172.21.15.103:3300/0 conn(0x7f90341375a0 msgr2=0x7f90340a9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:05.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.381+0000 7f9033fff640 1 --2- 172.21.15.103:0/3145624712 >> v2:172.21.15.103:3300/0 conn(0x7f90341375a0 0x7f90340a9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.381+0000 7f9033fff640 1 -- 172.21.15.103:0/3145624712 >> v1:172.21.15.103:6789/0 conn(0x7f903406c7e0 legacy=0x7f9034136e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.381+0000 7f9033fff640 1 -- 172.21.15.103:0/3145624712 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90340aa7e0 con 0x7f9034143f10 2024-08-19T13:19:05.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.381+0000 7f9032ffd640 1 --2- 172.21.15.103:0/3145624712 >> v2:172.21.15.103:3300/0 conn(0x7f90341375a0 0x7f90340a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:05.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.381+0000 7f9033fff640 1 --2- 172.21.15.103:0/3145624712 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9034143f10 0x7f90341443c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f902c009d90 tx=0x7f902c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e3dee988effb450 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:05.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.381+0000 7f90327fc640 1 -- 172.21.15.103:0/3145624712 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c031600 con 0x7f9034143f10 2024-08-19T13:19:05.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.381+0000 7f90327fc640 1 -- 172.21.15.103:0/3145624712 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f902c0317a0 con 0x7f9034143f10 2024-08-19T13:19:05.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.381+0000 7f90327fc640 1 -- 172.21.15.103:0/3145624712 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c031b70 con 0x7f9034143f10 2024-08-19T13:19:05.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.382+0000 7f9039df8640 1 -- 172.21.15.103:0/3145624712 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9034143f10 msgr2=0x7f90341443c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.382+0000 7f9039df8640 1 --2- 172.21.15.103:0/3145624712 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9034143f10 0x7f90341443c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f902c009d90 tx=0x7f902c02a5d0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.382+0000 7f9039df8640 1 -- 172.21.15.103:0/3145624712 shutdown_connections 2024-08-19T13:19:05.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.382+0000 7f9039df8640 1 --2- 172.21.15.103:0/3145624712 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9034143f10 0x7f90341443c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.382+0000 7f9039df8640 1 --2- 172.21.15.103:0/3145624712 >> v2:172.21.15.103:3300/0 conn(0x7f90341375a0 0x7f90340a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.382+0000 7f9039df8640 1 -- 172.21.15.103:0/3145624712 >> 172.21.15.103:0/3145624712 conn(0x7f90340681a0 msgr2=0x7f90340b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:05.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.382+0000 7f9039df8640 1 -- 172.21.15.103:0/3145624712 shutdown_connections 2024-08-19T13:19:05.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.382+0000 7f9039df8640 1 -- 172.21.15.103:0/3145624712 wait complete. 2024-08-19T13:19:05.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.383+0000 7f9039df8640 1 Processor -- start 2024-08-19T13:19:05.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.383+0000 7f9039df8640 1 -- start start 2024-08-19T13:19:05.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9039df8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f903406c7e0 0x7f90341d94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9039df8640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90341375a0 0x7f90341d99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9039df8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9034143f10 0x7f9034133280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9039df8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9034144d10 con 0x7f9034143f10 2024-08-19T13:19:05.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9039df8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9034144b90 con 0x7f903406c7e0 2024-08-19T13:19:05.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9039df8640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f903414b2a0 con 0x7f90341375a0 2024-08-19T13:19:05.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9032ffd640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90341375a0 0x7f90341d99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9033fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9034143f10 0x7f9034133280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9032ffd640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90341375a0 0x7f90341d99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:50802/0 (socket says 172.21.15.103:50802) 2024-08-19T13:19:05.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9032ffd640 1 -- 172.21.15.103:0/2186825051 learned_addr learned my addr 172.21.15.103:0/2186825051 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:05.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f90337fe640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f903406c7e0 0x7f90341d94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9033fff640 1 -- 172.21.15.103:0/2186825051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90341375a0 msgr2=0x7f90341d99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9033fff640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90341375a0 0x7f90341d99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9033fff640 1 -- 172.21.15.103:0/2186825051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f903406c7e0 msgr2=0x7f90341d94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9033fff640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f903406c7e0 0x7f90341d94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9033fff640 1 -- 172.21.15.103:0/2186825051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90341379a0 con 0x7f9034143f10 2024-08-19T13:19:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f90337fe640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f903406c7e0 0x7f90341d94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T13:19:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9032ffd640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90341375a0 0x7f90341d99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9033fff640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9034143f10 0x7f9034133280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f902c031000 tx=0x7f902c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f901bfff640 1 -- 172.21.15.103:0/2186825051 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c03f070 con 0x7f9034143f10 2024-08-19T13:19:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f901bfff640 1 -- 172.21.15.103:0/2186825051 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f902c030ba0 con 0x7f9034143f10 2024-08-19T13:19:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.384+0000 7f9039df8640 1 -- 172.21.15.103:0/2186825051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903413a970 con 0x7f9034143f10 2024-08-19T13:19:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.385+0000 7f901bfff640 1 -- 172.21.15.103:0/2186825051 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c009450 con 0x7f9034143f10 2024-08-19T13:19:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.385+0000 7f9039df8640 1 -- 172.21.15.103:0/2186825051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903413e540 con 0x7f9034143f10 2024-08-19T13:19:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.385+0000 7f9039df8640 1 -- 172.21.15.103:0/2186825051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90340a9c40 con 0x7f9034143f10 2024-08-19T13:19:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.386+0000 7f901bfff640 1 -- 172.21.15.103:0/2186825051 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f902c044460 con 0x7f9034143f10 2024-08-19T13:19:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.386+0000 7f901bfff640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f902006be60 0x7f902006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.386+0000 7f90337fe640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f902006be60 0x7f902006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.386+0000 7f901bfff640 1 -- 172.21.15.103:0/2186825051 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7f902c0ac220 con 0x7f9034143f10 2024-08-19T13:19:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.387+0000 7f90337fe640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f902006be60 0x7f902006e320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9024001680 tx=0x7f9024009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.391+0000 7f901bfff640 1 -- 172.21.15.103:0/2186825051 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f902c07aca0 con 0x7f9034143f10 2024-08-19T13:19:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.556+0000 7f9039df8640 1 -- 172.21.15.103:0/2186825051 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f903406b800 con 0x7f902006be60 2024-08-19T13:19:05.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.558+0000 7f901bfff640 1 -- 172.21.15.103:0/2186825051 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117854 (secure 0 0 0) 0x7f903406b800 con 0x7f902006be60 2024-08-19T13:19:05.560 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:19:05.560 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-08-19T13:19:05.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.561+0000 7f9039df8640 1 -- 172.21.15.103:0/2186825051 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f902006be60 msgr2=0x7f902006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.561+0000 7f9039df8640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f902006be60 0x7f902006e320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9024001680 tx=0x7f9024009040 comp rx=0 tx=0).stop 2024-08-19T13:19:05.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.561+0000 7f9039df8640 1 -- 172.21.15.103:0/2186825051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9034143f10 msgr2=0x7f9034133280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.561+0000 7f9039df8640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9034143f10 0x7f9034133280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f902c031000 tx=0x7f902c004a40 comp rx=0 tx=0).stop 2024-08-19T13:19:05.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.561+0000 7f9039df8640 1 -- 172.21.15.103:0/2186825051 shutdown_connections 2024-08-19T13:19:05.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.561+0000 7f9039df8640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f902006be60 0x7f902006e320 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.561+0000 7f9039df8640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9034143f10 0x7f9034133280 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.561+0000 7f9039df8640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90341375a0 0x7f90341d99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.561+0000 7f9039df8640 1 --2- 172.21.15.103:0/2186825051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f903406c7e0 0x7f90341d94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.561+0000 7f9039df8640 1 -- 172.21.15.103:0/2186825051 >> 172.21.15.103:0/2186825051 conn(0x7f90340681a0 msgr2=0x7f9034136170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:05.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.561+0000 7f9039df8640 1 -- 172.21.15.103:0/2186825051 shutdown_connections 2024-08-19T13:19:05.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.561+0000 7f9039df8640 1 -- 172.21.15.103:0/2186825051 wait complete. 2024-08-19T13:19:05.572 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-08-19T13:19:04.047415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3575960891,"num_objects":884,"num_object_clones":0,"num_object_copies":1768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":884,"num_whiteouts":0,"num_read":2623,"num_read_kb":2618,"num_write":162624,"num_write_kb":5650011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":165414,"ondisk_log_size":165414,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9309976,"kb_used_data":6675992,"kb_used_omap":12,"kb_used_meta":2633907,"kb_avail":745664744,"statfs":{"total":773094113280,"available":763560697856,"internally_reserved":0,"allocated":6836215808,"data_stored":6851292878,"data_compressed":70441838,"data_compressed_allocated":73416704,"data_compressed_original":94011392,"omap_allocated":12752,"internal_metadata":2697121328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40960,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002825"},"pg_stats":[{"pgid":"3.1f","version":"22'3551","reported_seq":3577,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.759901+0000","last_change":"2024-08-19T13:15:04.704168+0000","last_active":"2024-08-19T13:17:47.759901+0000","last_peered":"2024-08-19T13:17:47.759901+0000","last_clean":"2024-08-19T13:17:47.759901+0000","last_became_active":"2024-08-19T13:15:04.703897+0000","last_became_peered":"2024-08-19T13:15:04.703897+0000","last_unstale":"2024-08-19T13:17:47.759901+0000","last_undegraded":"2024-08-19T13:17:47.759901+0000","last_fullsized":"2024-08-19T13:17:47.759901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3551,"log_dups_size":0,"ondisk_log_size":3551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:54:14.098742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":3566,"num_write_kb":193154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'5748","reported_seq":5766,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:54.365605+0000","last_change":"2024-08-19T13:15:04.702034+0000","last_active":"2024-08-19T13:17:54.365605+0000","last_peered":"2024-08-19T13:17:54.365605+0000","last_clean":"2024-08-19T13:17:54.365605+0000","last_became_active":"2024-08-19T13:15:04.701653+0000","last_became_peered":"2024-08-19T13:15:04.701653+0000","last_unstale":"2024-08-19T13:17:54.365605+0000","last_undegraded":"2024-08-19T13:17:54.365605+0000","last_fullsized":"2024-08-19T13:17:54.365605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5748,"log_dups_size":0,"ondisk_log_size":5748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T15:23:44.488137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128583504,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5733,"num_write_kb":196004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"23'5874","reported_seq":5890,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:00.011644+0000","last_change":"2024-08-19T13:15:04.708047+0000","last_active":"2024-08-19T13:18:00.011644+0000","last_peered":"2024-08-19T13:18:00.011644+0000","last_clean":"2024-08-19T13:18:00.011644+0000","last_became_active":"2024-08-19T13:15:04.707756+0000","last_became_peered":"2024-08-19T13:15:04.707756+0000","last_unstale":"2024-08-19T13:18:00.011644+0000","last_undegraded":"2024-08-19T13:18:00.011644+0000","last_fullsized":"2024-08-19T13:18:00.011644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5874,"log_dups_size":0,"ondisk_log_size":5874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:23:19.211747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113312264,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5859,"num_write_kb":177354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"22'1998","reported_seq":2013,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.729767+0000","last_change":"2024-08-19T13:15:04.707039+0000","last_active":"2024-08-19T13:17:47.729767+0000","last_peered":"2024-08-19T13:17:47.729767+0000","last_clean":"2024-08-19T13:17:47.729767+0000","last_became_active":"2024-08-19T13:15:04.706726+0000","last_became_peered":"2024-08-19T13:15:04.706726+0000","last_unstale":"2024-08-19T13:17:47.729767+0000","last_undegraded":"2024-08-19T13:17:47.729767+0000","last_fullsized":"2024-08-19T13:17:47.729767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":1998,"log_dups_size":0,"ondisk_log_size":1998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:45:09.075439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1990,"num_write_kb":94721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"23'3781","reported_seq":3797,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.969389+0000","last_change":"2024-08-19T13:15:04.707093+0000","last_active":"2024-08-19T13:17:47.969389+0000","last_peered":"2024-08-19T13:17:47.969389+0000","last_clean":"2024-08-19T13:17:47.969389+0000","last_became_active":"2024-08-19T13:15:04.706832+0000","last_became_peered":"2024-08-19T13:15:04.706832+0000","last_unstale":"2024-08-19T13:17:47.969389+0000","last_undegraded":"2024-08-19T13:17:47.969389+0000","last_fullsized":"2024-08-19T13:17:47.969389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3781,"log_dups_size":0,"ondisk_log_size":3781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T15:22:00.995458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104858112,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3741,"num_write_kb":132744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"23'7047","reported_seq":7066,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:53.256999+0000","last_change":"2024-08-19T13:15:04.706943+0000","last_active":"2024-08-19T13:17:53.256999+0000","last_peered":"2024-08-19T13:17:53.256999+0000","last_clean":"2024-08-19T13:17:53.256999+0000","last_became_active":"2024-08-19T13:15:04.706371+0000","last_became_peered":"2024-08-19T13:15:04.706371+0000","last_unstale":"2024-08-19T13:17:53.256999+0000","last_undegraded":"2024-08-19T13:17:53.256999+0000","last_fullsized":"2024-08-19T13:17:53.256999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":7047,"log_dups_size":0,"ondisk_log_size":7047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:36:21.721116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134217728,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":7029,"num_write_kb":256635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"23'4609","reported_seq":4626,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:04.184395+0000","last_change":"2024-08-19T13:15:04.701266+0000","last_active":"2024-08-19T13:18:04.184395+0000","last_peered":"2024-08-19T13:18:04.184395+0000","last_clean":"2024-08-19T13:18:04.184395+0000","last_became_active":"2024-08-19T13:15:04.700993+0000","last_became_peered":"2024-08-19T13:15:04.700993+0000","last_unstale":"2024-08-19T13:18:04.184395+0000","last_undegraded":"2024-08-19T13:18:04.184395+0000","last_fullsized":"2024-08-19T13:18:04.184395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4609,"log_dups_size":0,"ondisk_log_size":4609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:01:09.023570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109052416,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":4571,"num_write_kb":180994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"23'6372","reported_seq":6387,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:04.058559+0000","last_change":"2024-08-19T13:15:04.699913+0000","last_active":"2024-08-19T13:18:04.058559+0000","last_peered":"2024-08-19T13:18:04.058559+0000","last_clean":"2024-08-19T13:18:04.058559+0000","last_became_active":"2024-08-19T13:15:04.699219+0000","last_became_peered":"2024-08-19T13:15:04.699219+0000","last_unstale":"2024-08-19T13:18:04.058559+0000","last_undegraded":"2024-08-19T13:18:04.058559+0000","last_fullsized":"2024-08-19T13:18:04.058559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":6372,"log_dups_size":0,"ondisk_log_size":6372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:00:52.367696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6366,"num_write_kb":136778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"23'7655","reported_seq":7673,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:07.987425+0000","last_change":"2024-08-19T13:15:04.701157+0000","last_active":"2024-08-19T13:18:07.987425+0000","last_peered":"2024-08-19T13:18:07.987425+0000","last_clean":"2024-08-19T13:18:07.987425+0000","last_became_active":"2024-08-19T13:15:04.700904+0000","last_became_peered":"2024-08-19T13:15:04.700904+0000","last_unstale":"2024-08-19T13:18:07.987425+0000","last_undegraded":"2024-08-19T13:18:07.987425+0000","last_fullsized":"2024-08-19T13:18:07.987425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":7655,"log_dups_size":0,"ondisk_log_size":7655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:41:45.337705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146932240,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":7658,"num_write_kb":230670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"23'3889","reported_seq":3904,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.917748+0000","last_change":"2024-08-19T13:15:04.704857+0000","last_active":"2024-08-19T13:17:47.917748+0000","last_peered":"2024-08-19T13:17:47.917748+0000","last_clean":"2024-08-19T13:17:47.917748+0000","last_became_active":"2024-08-19T13:15:04.704383+0000","last_became_peered":"2024-08-19T13:15:04.704383+0000","last_unstale":"2024-08-19T13:17:47.917748+0000","last_undegraded":"2024-08-19T13:17:47.917748+0000","last_fullsized":"2024-08-19T13:17:47.917748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3889,"log_dups_size":0,"ondisk_log_size":3889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:27:48.271684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106366632,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3864,"num_write_kb":162549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"23'8294","reported_seq":8332,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:03.337243+0000","last_change":"2024-08-19T13:15:04.703573+0000","last_active":"2024-08-19T13:18:03.337243+0000","last_peered":"2024-08-19T13:18:03.337243+0000","last_clean":"2024-08-19T13:18:03.337243+0000","last_became_active":"2024-08-19T13:15:04.703277+0000","last_became_peered":"2024-08-19T13:15:04.703277+0000","last_unstale":"2024-08-19T13:18:03.337243+0000","last_undegraded":"2024-08-19T13:18:03.337243+0000","last_fullsized":"2024-08-19T13:18:03.337243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":8294,"log_dups_size":0,"ondisk_log_size":8294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:08:33.844086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125960192,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":85,"num_read_kb":132,"num_write":8326,"num_write_kb":182642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"23'6457","reported_seq":6573,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:57.290424+0000","last_change":"2024-08-19T13:15:04.707221+0000","last_active":"2024-08-19T13:17:57.290424+0000","last_peered":"2024-08-19T13:17:57.290424+0000","last_clean":"2024-08-19T13:17:57.290424+0000","last_became_active":"2024-08-19T13:15:04.706496+0000","last_became_peered":"2024-08-19T13:15:04.706496+0000","last_unstale":"2024-08-19T13:17:57.290424+0000","last_undegraded":"2024-08-19T13:17:57.290424+0000","last_fullsized":"2024-08-19T13:17:57.290424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":6457,"log_dups_size":0,"ondisk_log_size":6457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:43:34.377186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90964496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":6419,"num_write_kb":175583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.730287+0000","last_change":"2024-08-19T13:14:58.168503+0000","last_active":"2024-08-19T13:17:47.730287+0000","last_peered":"2024-08-19T13:17:47.730287+0000","last_clean":"2024-08-19T13:17:47.730287+0000","last_became_active":"2024-08-19T13:14:56.044662+0000","last_became_peered":"2024-08-19T13:14:56.044662+0000","last_unstale":"2024-08-19T13:17:47.730287+0000","last_undegraded":"2024-08-19T13:17:47.730287+0000","last_fullsized":"2024-08-19T13:17:47.730287+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T23:26:23.570412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039371300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'6792","reported_seq":6816,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:09.561558+0000","last_change":"2024-08-19T13:15:04.700796+0000","last_active":"2024-08-19T13:18:09.561558+0000","last_peered":"2024-08-19T13:18:09.561558+0000","last_clean":"2024-08-19T13:18:09.561558+0000","last_became_active":"2024-08-19T13:15:04.700540+0000","last_became_peered":"2024-08-19T13:15:04.700540+0000","last_unstale":"2024-08-19T13:18:09.561558+0000","last_undegraded":"2024-08-19T13:18:09.561558+0000","last_fullsized":"2024-08-19T13:18:09.561558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":6792,"log_dups_size":0,"ondisk_log_size":6792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:39:03.153240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104988672,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":34,"num_read_kb":151,"num_write":6787,"num_write_kb":141259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.758067+0000","last_change":"2024-08-19T13:14:58.061578+0000","last_active":"2024-08-19T13:17:47.758067+0000","last_peered":"2024-08-19T13:17:47.758067+0000","last_clean":"2024-08-19T13:17:47.758067+0000","last_became_active":"2024-08-19T13:14:56.045731+0000","last_became_peered":"2024-08-19T13:14:56.045731+0000","last_unstale":"2024-08-19T13:17:47.758067+0000","last_undegraded":"2024-08-19T13:17:47.758067+0000","last_fullsized":"2024-08-19T13:17:47.758067+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:49:02.156480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044000000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'5379","reported_seq":5402,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.968669+0000","last_change":"2024-08-19T13:15:04.703333+0000","last_active":"2024-08-19T13:17:47.968669+0000","last_peered":"2024-08-19T13:17:47.968669+0000","last_clean":"2024-08-19T13:17:47.968669+0000","last_became_active":"2024-08-19T13:15:04.703011+0000","last_became_peered":"2024-08-19T13:15:04.703011+0000","last_unstale":"2024-08-19T13:17:47.968669+0000","last_undegraded":"2024-08-19T13:17:47.968669+0000","last_fullsized":"2024-08-19T13:17:47.968669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5379,"log_dups_size":0,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:08:52.537445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146276352,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5343,"num_write_kb":213509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.740138+0000","last_change":"2024-08-19T13:14:58.373877+0000","last_active":"2024-08-19T13:17:47.740138+0000","last_peered":"2024-08-19T13:17:47.740138+0000","last_clean":"2024-08-19T13:17:47.740138+0000","last_became_active":"2024-08-19T13:14:56.111427+0000","last_became_peered":"2024-08-19T13:14:56.111427+0000","last_unstale":"2024-08-19T13:17:47.740138+0000","last_undegraded":"2024-08-19T13:17:47.740138+0000","last_fullsized":"2024-08-19T13:17:47.740138+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:19:29.396593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050374000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'8281","reported_seq":8296,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:06.692281+0000","last_change":"2024-08-19T13:15:04.699221+0000","last_active":"2024-08-19T13:18:06.692281+0000","last_peered":"2024-08-19T13:18:06.692281+0000","last_clean":"2024-08-19T13:18:06.692281+0000","last_became_active":"2024-08-19T13:15:04.698944+0000","last_became_peered":"2024-08-19T13:15:04.698944+0000","last_unstale":"2024-08-19T13:18:06.692281+0000","last_undegraded":"2024-08-19T13:18:06.692281+0000","last_fullsized":"2024-08-19T13:18:06.692281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":8281,"log_dups_size":0,"ondisk_log_size":8281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:45:55.451611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8278,"num_write_kb":279553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.757967+0000","last_change":"2024-08-19T13:14:58.061576+0000","last_active":"2024-08-19T13:17:47.757967+0000","last_peered":"2024-08-19T13:17:47.757967+0000","last_clean":"2024-08-19T13:17:47.757967+0000","last_became_active":"2024-08-19T13:14:56.111065+0000","last_became_peered":"2024-08-19T13:14:56.111065+0000","last_unstale":"2024-08-19T13:17:47.757967+0000","last_undegraded":"2024-08-19T13:17:47.757967+0000","last_fullsized":"2024-08-19T13:17:47.757967+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:43:13.276725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042019699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'4283","reported_seq":4312,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:52.878133+0000","last_change":"2024-08-19T13:15:04.698014+0000","last_active":"2024-08-19T13:18:52.878133+0000","last_peered":"2024-08-19T13:18:52.878133+0000","last_clean":"2024-08-19T13:18:52.878133+0000","last_became_active":"2024-08-19T13:15:04.697675+0000","last_became_peered":"2024-08-19T13:15:04.697675+0000","last_unstale":"2024-08-19T13:18:52.878133+0000","last_undegraded":"2024-08-19T13:18:52.878133+0000","last_fullsized":"2024-08-19T13:18:52.878133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4283,"log_dups_size":0,"ondisk_log_size":4283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:28:29.123627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114253824,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":58,"num_read_kb":45,"num_write":4306,"num_write_kb":159578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'5377","reported_seq":5407,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.733133+0000","last_change":"2024-08-19T13:15:04.703723+0000","last_active":"2024-08-19T13:17:47.733133+0000","last_peered":"2024-08-19T13:17:47.733133+0000","last_clean":"2024-08-19T13:17:47.733133+0000","last_became_active":"2024-08-19T13:15:04.703417+0000","last_became_peered":"2024-08-19T13:15:04.703417+0000","last_unstale":"2024-08-19T13:17:47.733133+0000","last_undegraded":"2024-08-19T13:17:47.733133+0000","last_fullsized":"2024-08-19T13:17:47.733133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5377,"log_dups_size":0,"ondisk_log_size":5377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:11:03.088099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":5405,"num_write_kb":165854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.729507+0000","last_change":"2024-08-19T13:14:58.056552+0000","last_active":"2024-08-19T13:17:47.729507+0000","last_peered":"2024-08-19T13:17:47.729507+0000","last_clean":"2024-08-19T13:17:47.729507+0000","last_became_active":"2024-08-19T13:14:56.045753+0000","last_became_peered":"2024-08-19T13:14:56.045753+0000","last_unstale":"2024-08-19T13:17:47.729507+0000","last_undegraded":"2024-08-19T13:17:47.729507+0000","last_fullsized":"2024-08-19T13:17:47.729507+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:12:21.860038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037158599999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"23'2823","reported_seq":2843,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.965052+0000","last_change":"2024-08-19T13:15:04.699545+0000","last_active":"2024-08-19T13:17:47.965052+0000","last_peered":"2024-08-19T13:17:47.965052+0000","last_clean":"2024-08-19T13:17:47.965052+0000","last_became_active":"2024-08-19T13:15:04.699255+0000","last_became_peered":"2024-08-19T13:15:04.699255+0000","last_unstale":"2024-08-19T13:17:47.965052+0000","last_undegraded":"2024-08-19T13:17:47.965052+0000","last_fullsized":"2024-08-19T13:17:47.965052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":2823,"log_dups_size":0,"ondisk_log_size":2823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:46:06.725740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84017152,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":2784,"num_write_kb":135939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.736581+0000","last_change":"2024-08-19T13:14:58.055759+0000","last_active":"2024-08-19T13:17:47.736581+0000","last_peered":"2024-08-19T13:17:47.736581+0000","last_clean":"2024-08-19T13:17:47.736581+0000","last_became_active":"2024-08-19T13:14:56.046524+0000","last_became_peered":"2024-08-19T13:14:56.046524+0000","last_unstale":"2024-08-19T13:17:47.736581+0000","last_undegraded":"2024-08-19T13:17:47.736581+0000","last_fullsized":"2024-08-19T13:17:47.736581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T23:05:11.782761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043587900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"23'5930","reported_seq":5946,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.915146+0000","last_change":"2024-08-19T13:15:04.710024+0000","last_active":"2024-08-19T13:17:47.915146+0000","last_peered":"2024-08-19T13:17:47.915146+0000","last_clean":"2024-08-19T13:17:47.915146+0000","last_became_active":"2024-08-19T13:15:04.709768+0000","last_became_peered":"2024-08-19T13:15:04.709768+0000","last_unstale":"2024-08-19T13:17:47.915146+0000","last_undegraded":"2024-08-19T13:17:47.915146+0000","last_fullsized":"2024-08-19T13:17:47.915146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5930,"log_dups_size":0,"ondisk_log_size":5930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:26:22.787995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146866696,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5915,"num_write_kb":254104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.739748+0000","last_change":"2024-08-19T13:14:58.373682+0000","last_active":"2024-08-19T13:17:47.739748+0000","last_peered":"2024-08-19T13:17:47.739748+0000","last_clean":"2024-08-19T13:17:47.739748+0000","last_became_active":"2024-08-19T13:14:56.045391+0000","last_became_peered":"2024-08-19T13:14:56.045391+0000","last_unstale":"2024-08-19T13:17:47.739748+0000","last_undegraded":"2024-08-19T13:17:47.739748+0000","last_fullsized":"2024-08-19T13:17:47.739748+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:41:33.266324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037776800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"23'5375","reported_seq":5407,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:01.600150+0000","last_change":"2024-08-19T13:15:04.698787+0000","last_active":"2024-08-19T13:18:01.600150+0000","last_peered":"2024-08-19T13:18:01.600150+0000","last_clean":"2024-08-19T13:18:01.600150+0000","last_became_active":"2024-08-19T13:15:04.698015+0000","last_became_peered":"2024-08-19T13:15:04.698015+0000","last_unstale":"2024-08-19T13:18:01.600150+0000","last_undegraded":"2024-08-19T13:18:01.600150+0000","last_fullsized":"2024-08-19T13:18:01.600150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5375,"log_dups_size":0,"ondisk_log_size":5375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:07:36.161135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":68,"num_read_kb":114,"num_write":5405,"num_write_kb":156434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.729454+0000","last_change":"2024-08-19T13:14:58.056578+0000","last_active":"2024-08-19T13:17:47.729454+0000","last_peered":"2024-08-19T13:17:47.729454+0000","last_clean":"2024-08-19T13:17:47.729454+0000","last_became_active":"2024-08-19T13:14:56.045464+0000","last_became_peered":"2024-08-19T13:14:56.045464+0000","last_unstale":"2024-08-19T13:17:47.729454+0000","last_undegraded":"2024-08-19T13:17:47.729454+0000","last_fullsized":"2024-08-19T13:17:47.729454+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:35:16.028455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039910299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.739769+0000","last_change":"2024-08-19T13:14:55.043964+0000","last_active":"2024-08-19T13:17:47.739769+0000","last_peered":"2024-08-19T13:17:47.739769+0000","last_clean":"2024-08-19T13:17:47.739769+0000","last_became_active":"2024-08-19T13:14:55.043626+0000","last_became_peered":"2024-08-19T13:14:55.043626+0000","last_unstale":"2024-08-19T13:17:47.739769+0000","last_undegraded":"2024-08-19T13:17:47.739769+0000","last_fullsized":"2024-08-19T13:17:47.739769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:54.027963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:54.027963+0000","last_clean_scrub_stamp":"2024-08-19T13:14:54.027963+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:36:05.876096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"23'2767","reported_seq":2782,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.965472+0000","last_change":"2024-08-19T13:15:04.699203+0000","last_active":"2024-08-19T13:17:47.965472+0000","last_peered":"2024-08-19T13:17:47.965472+0000","last_clean":"2024-08-19T13:17:47.965472+0000","last_became_active":"2024-08-19T13:15:04.698903+0000","last_became_peered":"2024-08-19T13:15:04.698903+0000","last_unstale":"2024-08-19T13:17:47.965472+0000","last_undegraded":"2024-08-19T13:17:47.965472+0000","last_fullsized":"2024-08-19T13:17:47.965472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:32:06.151208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2699,"num_write_kb":115587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"23'9933","reported_seq":10062,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:04.629618+0000","last_change":"2024-08-19T13:15:04.708930+0000","last_active":"2024-08-19T13:18:04.629618+0000","last_peered":"2024-08-19T13:18:04.629618+0000","last_clean":"2024-08-19T13:18:04.629618+0000","last_became_active":"2024-08-19T13:15:04.706442+0000","last_became_peered":"2024-08-19T13:15:04.706442+0000","last_unstale":"2024-08-19T13:18:04.629618+0000","last_undegraded":"2024-08-19T13:18:04.629618+0000","last_fullsized":"2024-08-19T13:18:04.629618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":9933,"log_dups_size":0,"ondisk_log_size":9933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:18:30.218698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145424384,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":361,"num_read_kb":322,"num_write":6867,"num_write_kb":212554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"23'5336","reported_seq":5354,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:18:05.747745+0000","last_change":"2024-08-19T13:15:04.708013+0000","last_active":"2024-08-19T13:18:05.747745+0000","last_peered":"2024-08-19T13:18:05.747745+0000","last_clean":"2024-08-19T13:18:05.747745+0000","last_became_active":"2024-08-19T13:15:04.707724+0000","last_became_peered":"2024-08-19T13:15:04.707724+0000","last_unstale":"2024-08-19T13:18:05.747745+0000","last_undegraded":"2024-08-19T13:18:05.747745+0000","last_fullsized":"2024-08-19T13:18:05.747745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:59:28.795104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":11,"num_read_kb":71,"num_write":5419,"num_write_kb":161478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"22'3547","reported_seq":3562,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.729697+0000","last_change":"2024-08-19T13:15:04.700346+0000","last_active":"2024-08-19T13:17:47.729697+0000","last_peered":"2024-08-19T13:17:47.729697+0000","last_clean":"2024-08-19T13:17:47.729697+0000","last_became_active":"2024-08-19T13:15:04.699228+0000","last_became_peered":"2024-08-19T13:15:04.699228+0000","last_unstale":"2024-08-19T13:17:47.729697+0000","last_undegraded":"2024-08-19T13:17:47.729697+0000","last_fullsized":"2024-08-19T13:17:47.729697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3547,"log_dups_size":0,"ondisk_log_size":3547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T23:29:48.186111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3536,"num_write_kb":144645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"23'3397","reported_seq":3441,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.967033+0000","last_change":"2024-08-19T13:15:04.699676+0000","last_active":"2024-08-19T13:17:47.967033+0000","last_peered":"2024-08-19T13:17:47.967033+0000","last_clean":"2024-08-19T13:17:47.967033+0000","last_became_active":"2024-08-19T13:15:04.699383+0000","last_became_peered":"2024-08-19T13:15:04.699383+0000","last_unstale":"2024-08-19T13:17:47.967033+0000","last_undegraded":"2024-08-19T13:17:47.967033+0000","last_fullsized":"2024-08-19T13:17:47.967033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3397,"log_dups_size":0,"ondisk_log_size":3397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:10:11.285767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":121,"num_read_kb":99,"num_write":3411,"num_write_kb":138057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'4376","reported_seq":4398,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.916399+0000","last_change":"2024-08-19T13:15:04.709245+0000","last_active":"2024-08-19T13:17:47.916399+0000","last_peered":"2024-08-19T13:17:47.916399+0000","last_clean":"2024-08-19T13:17:47.916399+0000","last_became_active":"2024-08-19T13:15:04.708174+0000","last_became_peered":"2024-08-19T13:15:04.708174+0000","last_unstale":"2024-08-19T13:17:47.916399+0000","last_undegraded":"2024-08-19T13:17:47.916399+0000","last_fullsized":"2024-08-19T13:17:47.916399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4376,"log_dups_size":0,"ondisk_log_size":4376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:25:57.887618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123076608,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":27,"num_read_kb":82,"num_write":4364,"num_write_kb":209707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'4908","reported_seq":5001,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:48.004070+0000","last_change":"2024-08-19T13:15:04.707739+0000","last_active":"2024-08-19T13:17:48.004070+0000","last_peered":"2024-08-19T13:17:48.004070+0000","last_clean":"2024-08-19T13:17:48.004070+0000","last_became_active":"2024-08-19T13:15:04.707482+0000","last_became_peered":"2024-08-19T13:15:04.707482+0000","last_unstale":"2024-08-19T13:17:48.004070+0000","last_undegraded":"2024-08-19T13:17:48.004070+0000","last_fullsized":"2024-08-19T13:17:48.004070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4908,"log_dups_size":0,"ondisk_log_size":4908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:07:38.674944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123349040,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":288,"num_read_kb":216,"num_write":5188,"num_write_kb":204415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'3580","reported_seq":3595,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.969742+0000","last_change":"2024-08-19T13:15:04.707186+0000","last_active":"2024-08-19T13:17:47.969742+0000","last_peered":"2024-08-19T13:17:47.969742+0000","last_clean":"2024-08-19T13:17:47.969742+0000","last_became_active":"2024-08-19T13:15:04.706891+0000","last_became_peered":"2024-08-19T13:15:04.706891+0000","last_unstale":"2024-08-19T13:17:47.969742+0000","last_undegraded":"2024-08-19T13:17:47.969742+0000","last_fullsized":"2024-08-19T13:17:47.969742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3580,"log_dups_size":0,"ondisk_log_size":3580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:45:10.115661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114560672,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3532,"num_write_kb":190257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"23'2893","reported_seq":2908,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.966115+0000","last_change":"2024-08-19T13:15:04.704399+0000","last_active":"2024-08-19T13:17:47.966115+0000","last_peered":"2024-08-19T13:17:47.966115+0000","last_clean":"2024-08-19T13:17:47.966115+0000","last_became_active":"2024-08-19T13:15:04.704113+0000","last_became_peered":"2024-08-19T13:15:04.704113+0000","last_unstale":"2024-08-19T13:17:47.966115+0000","last_undegraded":"2024-08-19T13:17:47.966115+0000","last_fullsized":"2024-08-19T13:17:47.966115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":2893,"log_dups_size":0,"ondisk_log_size":2893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:09:37.265840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103415808,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2845,"num_write_kb":157315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"22'4138","reported_seq":4158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.739708+0000","last_change":"2024-08-19T13:15:04.709668+0000","last_active":"2024-08-19T13:17:47.739708+0000","last_peered":"2024-08-19T13:17:47.739708+0000","last_clean":"2024-08-19T13:17:47.739708+0000","last_became_active":"2024-08-19T13:15:04.709376+0000","last_became_peered":"2024-08-19T13:15:04.709376+0000","last_unstale":"2024-08-19T13:17:47.739708+0000","last_undegraded":"2024-08-19T13:17:47.739708+0000","last_fullsized":"2024-08-19T13:17:47.739708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:26:41.219376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":4141,"num_write_kb":181515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"23'3049","reported_seq":3079,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:47.966674+0000","last_change":"2024-08-19T13:15:04.704226+0000","last_active":"2024-08-19T13:17:47.966674+0000","last_peered":"2024-08-19T13:17:47.966674+0000","last_clean":"2024-08-19T13:17:47.966674+0000","last_became_active":"2024-08-19T13:15:04.703965+0000","last_became_peered":"2024-08-19T13:15:04.703965+0000","last_unstale":"2024-08-19T13:17:47.966674+0000","last_undegraded":"2024-08-19T13:17:47.966674+0000","last_fullsized":"2024-08-19T13:17:47.966674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3049,"log_dups_size":0,"ondisk_log_size":3049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:51:52.289558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79700496,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":58,"num_read_kb":46,"num_write":3047,"num_write_kb":132115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"23'7855","reported_seq":8198,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-08-19T13:17:59.958424+0000","last_change":"2024-08-19T13:15:04.707304+0000","last_active":"2024-08-19T13:17:59.958424+0000","last_peered":"2024-08-19T13:17:59.958424+0000","last_clean":"2024-08-19T13:17:59.958424+0000","last_became_active":"2024-08-19T13:15:04.706952+0000","last_became_peered":"2024-08-19T13:15:04.706952+0000","last_unstale":"2024-08-19T13:17:59.958424+0000","last_undegraded":"2024-08-19T13:17:59.958424+0000","last_fullsized":"2024-08-19T13:17:59.958424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":7855,"log_dups_size":0,"ondisk_log_size":7855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:27:03.627133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1110,"num_read_kb":992,"num_write":8069,"num_write_kb":173890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3575370504,"num_objects":880,"num_object_clones":0,"num_object_copies":1760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":880,"num_whiteouts":0,"num_read":2507,"num_read_kb":2518,"num_write":162463,"num_write_kb":5647593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6834151424,"data_stored":6849530008,"data_compressed":70382846,"data_compressed_allocated":73334784,"data_compressed_original":92987392,"omap_allocated":0,"internal_metadata":0},"log_size":165294,"ondisk_log_size":165294,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":58992,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1448372,"kb_used_data":1016044,"kb_used_omap":1,"kb_used_meta":432318,"kb_avail":92923468,"statfs":{"total":96636764160,"available":95153631232,"internally_reserved":0,"allocated":1040429056,"data_stored":1041943846,"data_compressed":9088911,"data_compressed_allocated":9474048,"data_compressed_original":11792384,"omap_allocated":1590,"internal_metadata":442694090},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443108,"kb_used_data":335580,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":93928732,"statfs":{"total":96636764160,"available":96183021568,"internally_reserved":0,"allocated":343633920,"data_stored":344904662,"data_compressed":5573327,"data_compressed_allocated":5816320,"data_compressed_original":7417856,"omap_allocated":1594,"internal_metadata":110098886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1250856,"kb_used_data":930976,"kb_used_omap":1,"kb_used_meta":319870,"kb_avail":93120984,"statfs":{"total":96636764160,"available":95355887616,"internally_reserved":0,"allocated":953319424,"data_stored":957661653,"data_compressed":10069671,"data_compressed_allocated":10522624,"data_compressed_original":15568896,"omap_allocated":1595,"internal_metadata":327547333},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1865984,"kb_used_data":1357368,"kb_used_omap":1,"kb_used_meta":508606,"kb_avail":92505856,"statfs":{"total":96636764160,"available":94725996544,"internally_reserved":0,"allocated":1389944832,"data_stored":1392673978,"data_compressed":17106045,"data_compressed_allocated":17801216,"data_compressed_original":21614592,"omap_allocated":1595,"internal_metadata":520812997},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1480504,"kb_used_data":1056496,"kb_used_omap":1,"kb_used_meta":423998,"kb_avail":92891336,"statfs":{"total":96636764160,"available":95120728064,"internally_reserved":0,"allocated":1081851904,"data_stored":1083347674,"data_compressed":11134625,"data_compressed_allocated":11599872,"data_compressed_original":13860864,"omap_allocated":1593,"internal_metadata":434174407},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496100,"kb_used_data":338652,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":93875740,"statfs":{"total":96636764160,"available":96128757760,"internally_reserved":0,"allocated":346779648,"data_stored":346707202,"data_compressed":1288748,"data_compressed_allocated":1335296,"data_compressed_original":1609728,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":874888,"kb_used_data":600832,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":93496952,"statfs":{"total":96636764160,"available":95740878848,"internally_reserved":0,"allocated":615251968,"data_stored":615348941,"data_compressed":3731208,"data_compressed_allocated":3866624,"data_compressed_original":4513792,"omap_allocated":1595,"internal_metadata":280623557},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1450164,"kb_used_data":1040044,"kb_used_omap":1,"kb_used_meta":410110,"kb_avail":92921676,"statfs":{"total":96636764160,"available":95151796224,"internally_reserved":0,"allocated":1065005056,"data_stored":1068704922,"data_compressed":12449303,"data_compressed_allocated":13000704,"data_compressed_original":17633280,"omap_allocated":1595,"internal_metadata":419953093},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1064701952,"data_stored":1068087872,"data_compressed":12419807,"data_compressed_allocated":12959744,"data_compressed_original":17121280,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":615006208,"data_stored":615260800,"data_compressed":3731208,"data_compressed_allocated":3866624,"data_compressed_original":4513792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346537984,"data_stored":346619080,"data_compressed":1288748,"data_compressed_allocated":1335296,"data_compressed_original":1609728,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1081610240,"data_stored":1083259552,"data_compressed":11134625,"data_compressed_allocated":11599872,"data_compressed_original":13860864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1389703168,"data_stored":1392585856,"data_compressed":17106045,"data_compressed_allocated":17801216,"data_compressed_original":21614592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":953073664,"data_stored":957573512,"data_compressed":10069671,"data_compressed_allocated":10522624,"data_compressed_original":15568896,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":343392256,"data_stored":344816540,"data_compressed":5573327,"data_compressed_allocated":5816320,"data_compressed_original":7417856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1040125952,"data_stored":1041326796,"data_compressed":9059415,"data_compressed_allocated":9433088,"data_compressed_original":11280384,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T13:19:05.575 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-19T13:19:05.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.672+0000 7f9d723bf640 1 Processor -- start 2024-08-19T13:19:05.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.672+0000 7f9d723bf640 1 -- start start 2024-08-19T13:19:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.672+0000 7f9d723bf640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9d6c072a40 0x7f9d6c073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.672+0000 7f9d723bf640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d6c142090 0x7f9d6c1424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.672+0000 7f9d723bf640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9d6c070240 0x7f9d6c0706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.672+0000 7f9d723bf640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f9d6c073640 con 0x7f9d6c072a40 2024-08-19T13:19:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.672+0000 7f9d723bf640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f9d6c0731b0 con 0x7f9d6c070240 2024-08-19T13:19:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.672+0000 7f9d723bf640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9d6c0b2f10 con 0x7f9d6c142090 2024-08-19T13:19:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.672+0000 7f9d6bfff640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9d6c072a40 0x7f9d6c073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.672+0000 7f9d6b7fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d6c142090 0x7f9d6c1424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.672+0000 7f9d6bfff640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9d6c072a40 0x7f9d6c073a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47478/0 (socket says 172.21.15.103:47478) 2024-08-19T13:19:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.672+0000 7f9d6b7fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d6c142090 0x7f9d6c1424d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:50808/0 (socket says 172.21.15.103:50808) 2024-08-19T13:19:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.672+0000 7f9d6bfff640 1 -- 172.21.15.103:0/691016462 learned_addr learned my addr 172.21.15.103:0/691016462 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.673+0000 7f9d7095c640 1 --2- 172.21.15.103:0/691016462 >> v2:172.21.15.122:3300/0 conn(0x7f9d6c070240 0x7f9d6c0706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.673+0000 7f9d6bfff640 1 -- 172.21.15.103:0/691016462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d6c142090 msgr2=0x7f9d6c1424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.673+0000 7f9d6bfff640 1 --2- 172.21.15.103:0/691016462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d6c142090 0x7f9d6c1424d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.673+0000 7f9d6bfff640 1 -- 172.21.15.103:0/691016462 >> v2:172.21.15.122:3300/0 conn(0x7f9d6c070240 msgr2=0x7f9d6c0706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.673+0000 7f9d6bfff640 1 --2- 172.21.15.103:0/691016462 >> v2:172.21.15.122:3300/0 conn(0x7f9d6c070240 0x7f9d6c0706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.673+0000 7f9d6bfff640 1 -- 172.21.15.103:0/691016462 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6c073f60 con 0x7f9d6c072a40 2024-08-19T13:19:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.673+0000 7f9d6b7fe640 1 --2- 172.21.15.103:0/691016462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d6c142090 0x7f9d6c1424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.673+0000 7f9d6bfff640 1 --2- 172.21.15.103:0/691016462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c072a40 0x7f9d6c073a20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0718a0 tx=0x7f9d5c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89280aea232b3ddf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:05.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.674+0000 7f9d6affd640 1 -- 172.21.15.103:0/691016462 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c037070 con 0x7f9d6c072a40 2024-08-19T13:19:05.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.674+0000 7f9d6affd640 1 -- 172.21.15.103:0/691016462 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d5c032440 con 0x7f9d6c072a40 2024-08-19T13:19:05.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.674+0000 7f9d6affd640 1 -- 172.21.15.103:0/691016462 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c032810 con 0x7f9d6c072a40 2024-08-19T13:19:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.674+0000 7f9d723bf640 1 -- 172.21.15.103:0/691016462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c072a40 msgr2=0x7f9d6c073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.674+0000 7f9d723bf640 1 --2- 172.21.15.103:0/691016462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c072a40 0x7f9d6c073a20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0718a0 tx=0x7f9d5c02a140 comp rx=0 tx=0).stop 2024-08-19T13:19:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.674+0000 7f9d723bf640 1 -- 172.21.15.103:0/691016462 shutdown_connections 2024-08-19T13:19:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.674+0000 7f9d723bf640 1 --2- 172.21.15.103:0/691016462 >> v2:172.21.15.122:3300/0 conn(0x7f9d6c070240 0x7f9d6c0706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.674+0000 7f9d723bf640 1 --2- 172.21.15.103:0/691016462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d6c142090 0x7f9d6c1424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.674+0000 7f9d723bf640 1 --2- 172.21.15.103:0/691016462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c072a40 0x7f9d6c073a20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.674+0000 7f9d723bf640 1 -- 172.21.15.103:0/691016462 >> 172.21.15.103:0/691016462 conn(0x7f9d6c0b8c90 msgr2=0x7f9d6c0b90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.674+0000 7f9d723bf640 1 -- 172.21.15.103:0/691016462 shutdown_connections 2024-08-19T13:19:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.674+0000 7f9d723bf640 1 -- 172.21.15.103:0/691016462 wait complete. 2024-08-19T13:19:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.675+0000 7f9d723bf640 1 Processor -- start 2024-08-19T13:19:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.675+0000 7f9d723bf640 1 -- start start 2024-08-19T13:19:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d723bf640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d6c070240 0x7f9d6c146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d723bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6c072a40 0x7f9d6c13c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d723bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c142090 0x7f9d6c13add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d723bf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c06fa40 con 0x7f9d6c142090 2024-08-19T13:19:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d723bf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c142ac0 con 0x7f9d6c072a40 2024-08-19T13:19:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d723bf640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9d6c070c30 con 0x7f9d6c070240 2024-08-19T13:19:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d6bfff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d6c070240 0x7f9d6c146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d7095c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c142090 0x7f9d6c13add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d6bfff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d6c070240 0x7f9d6c146a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:50816/0 (socket says 172.21.15.103:50816) 2024-08-19T13:19:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d6bfff640 1 -- 172.21.15.103:0/262733199 learned_addr learned my addr 172.21.15.103:0/262733199 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d7095c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c142090 0x7f9d6c13add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47492/0 (socket says 172.21.15.103:47492) 2024-08-19T13:19:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d6b7fe640 1 --2- 172.21.15.103:0/262733199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6c072a40 0x7f9d6c13c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d6bfff640 1 -- 172.21.15.103:0/262733199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6c072a40 msgr2=0x7f9d6c13c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d6bfff640 1 --2- 172.21.15.103:0/262733199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6c072a40 0x7f9d6c13c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d6bfff640 1 -- 172.21.15.103:0/262733199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c142090 msgr2=0x7f9d6c13add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d6bfff640 1 --2- 172.21.15.103:0/262733199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c142090 0x7f9d6c13add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d6bfff640 1 -- 172.21.15.103:0/262733199 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6c13c230 con 0x7f9d6c070240 2024-08-19T13:19:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d7095c640 1 --2- 172.21.15.103:0/262733199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c142090 0x7f9d6c13add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.676+0000 7f9d6bfff640 1 --2- 172.21.15.103:0/262733199 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d6c070240 0x7f9d6c146a80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c02a180 tx=0x7f9d5c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.677+0000 7f9d68ff9640 1 -- 172.21.15.103:0/262733199 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c037040 con 0x7f9d6c070240 2024-08-19T13:19:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.677+0000 7f9d68ff9640 1 -- 172.21.15.103:0/262733199 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d5c042070 con 0x7f9d6c070240 2024-08-19T13:19:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.677+0000 7f9d723bf640 1 -- 172.21.15.103:0/262733199 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6c13cee0 con 0x7f9d6c070240 2024-08-19T13:19:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.677+0000 7f9d68ff9640 1 -- 172.21.15.103:0/262733199 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c030860 con 0x7f9d6c070240 2024-08-19T13:19:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.677+0000 7f9d723bf640 1 -- 172.21.15.103:0/262733199 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d6c0b2510 con 0x7f9d6c070240 2024-08-19T13:19:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.678+0000 7f9d723bf640 1 -- 172.21.15.103:0/262733199 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d6c073a20 con 0x7f9d6c070240 2024-08-19T13:19:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.678+0000 7f9d68ff9640 1 -- 172.21.15.103:0/262733199 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d5c030a00 con 0x7f9d6c070240 2024-08-19T13:19:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.678+0000 7f9d68ff9640 1 --2- 172.21.15.103:0/262733199 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f9d5806be30 0x7f9d5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.678+0000 7f9d68ff9640 1 -- 172.21.15.103:0/262733199 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7f9d5c0ab4b0 con 0x7f9d6c070240 2024-08-19T13:19:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.679+0000 7f9d6b7fe640 1 --2- 172.21.15.103:0/262733199 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f9d5806be30 0x7f9d5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.679+0000 7f9d6b7fe640 1 --2- 172.21.15.103:0/262733199 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f9d5806be30 0x7f9d5806e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009500 tx=0x7f9d60009110 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.683+0000 7f9d68ff9640 1 -- 172.21.15.103:0/262733199 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f9d5c0798e0 con 0x7f9d6c070240 2024-08-19T13:19:05.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.848+0000 7f9d723bf640 1 -- 172.21.15.103:0/262733199 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d6c0b3db0 con 0x7f9d6c070240 2024-08-19T13:19:05.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.849+0000 7f9d68ff9640 1 -- 172.21.15.103:0/262733199 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16645 (secure 0 0 0) 0x7f9d5c07dbe0 con 0x7f9d6c070240 2024-08-19T13:19:05.850 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:19:05.850 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":24,"fsid":"9a6e456b-d04b-48cb-846f-0fbd1ee79f13","created":"2024-08-19T13:14:43.897750+0000","modified":"2024-08-19T13:19:03.853583+0000","last_up_change":"2024-08-19T13:14:53.010492+0000","last_in_change":"2024-08-19T13:14:46.047417+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T13:14:54.027206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-08-19T13:14:54.689859+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephsqlite","create_time":"2024-08-19T13:15:02.938690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01d9bd23-e3e8-4905-af40-142b43a27628","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6809","nonce":1798955708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6811","nonce":1798955708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6815","nonce":1798955708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1798955708},{"type":"v1","addr":"172.21.15.103:6813","nonce":1798955708}]},"public_addr":"172.21.15.103:6809/1798955708","cluster_addr":"172.21.15.103:6811/1798955708","heartbeat_back_addr":"172.21.15.103:6815/1798955708","heartbeat_front_addr":"172.21.15.103:6813/1798955708","state":["exists","up"]},{"osd":1,"uuid":"d562d5f9-b0ac-4f9c-856d-cdcf1be3da18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6825","nonce":855529591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6827","nonce":855529591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6831","nonce":855529591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":855529591},{"type":"v1","addr":"172.21.15.103:6829","nonce":855529591}]},"public_addr":"172.21.15.103:6825/855529591","cluster_addr":"172.21.15.103:6827/855529591","heartbeat_back_addr":"172.21.15.103:6831/855529591","heartbeat_front_addr":"172.21.15.103:6829/855529591","state":["exists","up"]},{"osd":2,"uuid":"adc47b95-8fbe-48c0-bb44-f78ba8ed6d29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6801","nonce":4004268210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6803","nonce":4004268210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6807","nonce":4004268210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4004268210},{"type":"v1","addr":"172.21.15.103:6805","nonce":4004268210}]},"public_addr":"172.21.15.103:6801/4004268210","cluster_addr":"172.21.15.103:6803/4004268210","heartbeat_back_addr":"172.21.15.103:6807/4004268210","heartbeat_front_addr":"172.21.15.103:6805/4004268210","state":["exists","up"]},{"osd":3,"uuid":"a2e730c7-32c8-4387-bdee-e00a70e7613a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6817","nonce":264232662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6819","nonce":264232662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6823","nonce":264232662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":264232662},{"type":"v1","addr":"172.21.15.103:6821","nonce":264232662}]},"public_addr":"172.21.15.103:6817/264232662","cluster_addr":"172.21.15.103:6819/264232662","heartbeat_back_addr":"172.21.15.103:6823/264232662","heartbeat_front_addr":"172.21.15.103:6821/264232662","state":["exists","up"]},{"osd":4,"uuid":"45b5502e-ce4c-411c-b7a2-1742483a432f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6801","nonce":634629443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6803","nonce":634629443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6807","nonce":634629443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":634629443},{"type":"v1","addr":"172.21.15.122:6805","nonce":634629443}]},"public_addr":"172.21.15.122:6801/634629443","cluster_addr":"172.21.15.122:6803/634629443","heartbeat_back_addr":"172.21.15.122:6807/634629443","heartbeat_front_addr":"172.21.15.122:6805/634629443","state":["exists","up"]},{"osd":5,"uuid":"056397bb-673f-461f-87c8-e44390d7ba37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6817","nonce":2112974081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6819","nonce":2112974081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6823","nonce":2112974081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2112974081},{"type":"v1","addr":"172.21.15.122:6821","nonce":2112974081}]},"public_addr":"172.21.15.122:6817/2112974081","cluster_addr":"172.21.15.122:6819/2112974081","heartbeat_back_addr":"172.21.15.122:6823/2112974081","heartbeat_front_addr":"172.21.15.122:6821/2112974081","state":["exists","up"]},{"osd":6,"uuid":"fab2d2d2-c670-4f56-b099-ca4686395179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6809","nonce":2033312647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6811","nonce":2033312647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6815","nonce":2033312647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2033312647},{"type":"v1","addr":"172.21.15.122:6813","nonce":2033312647}]},"public_addr":"172.21.15.122:6809/2033312647","cluster_addr":"172.21.15.122:6811/2033312647","heartbeat_back_addr":"172.21.15.122:6815/2033312647","heartbeat_front_addr":"172.21.15.122:6813/2033312647","state":["exists","up"]},{"osd":7,"uuid":"ee58594c-242f-436e-9c9e-977e57b6785f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6825","nonce":3566117434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6827","nonce":3566117434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6831","nonce":3566117434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3566117434},{"type":"v1","addr":"172.21.15.122:6829","nonce":3566117434}]},"public_addr":"172.21.15.122:6825/3566117434","cluster_addr":"172.21.15.122:6827/3566117434","heartbeat_back_addr":"172.21.15.122:6831/3566117434","heartbeat_front_addr":"172.21.15.122:6829/3566117434","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.192109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.184240+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.258392+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.144042+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.137671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.217965+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.194891+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T13:14:51.241799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/467671781":"2024-08-19T14:19:02.907915+0000","172.21.15.103:0/2454626700":"2024-08-19T14:17:47.318844+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T13:19:05.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.851+0000 7f9d723bf640 1 -- 172.21.15.103:0/262733199 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f9d5806be30 msgr2=0x7f9d5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.851+0000 7f9d723bf640 1 --2- 172.21.15.103:0/262733199 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f9d5806be30 0x7f9d5806e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009500 tx=0x7f9d60009110 comp rx=0 tx=0).stop 2024-08-19T13:19:05.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.851+0000 7f9d723bf640 1 -- 172.21.15.103:0/262733199 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d6c070240 msgr2=0x7f9d6c146a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:05.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.851+0000 7f9d723bf640 1 --2- 172.21.15.103:0/262733199 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d6c070240 0x7f9d6c146a80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c02a180 tx=0x7f9d5c0046c0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.851+0000 7f9d723bf640 1 -- 172.21.15.103:0/262733199 shutdown_connections 2024-08-19T13:19:05.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.851+0000 7f9d723bf640 1 --2- 172.21.15.103:0/262733199 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f9d5806be30 0x7f9d5806e2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.851+0000 7f9d723bf640 1 --2- 172.21.15.103:0/262733199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c142090 0x7f9d6c13add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.851+0000 7f9d723bf640 1 --2- 172.21.15.103:0/262733199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6c072a40 0x7f9d6c13c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.851+0000 7f9d723bf640 1 --2- 172.21.15.103:0/262733199 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d6c070240 0x7f9d6c146a80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:05.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.851+0000 7f9d723bf640 1 -- 172.21.15.103:0/262733199 >> 172.21.15.103:0/262733199 conn(0x7f9d6c0b8c90 msgr2=0x7f9d6c0b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:05.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.851+0000 7f9d723bf640 1 -- 172.21.15.103:0/262733199 shutdown_connections 2024-08-19T13:19:05.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:05.851+0000 7f9d723bf640 1 -- 172.21.15.103:0/262733199 wait complete. 2024-08-19T13:19:06.863 INFO:tasks.ceph:Scrubbing osd.0 2024-08-19T13:19:06.864 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-08-19T13:19:06.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.974+0000 7f3fa5abe640 1 Processor -- start 2024-08-19T13:19:06.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.974+0000 7f3fa5abe640 1 -- start start 2024-08-19T13:19:06.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.975+0000 7f3fa5abe640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3fa01437d0 0x7f3fa014b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:06.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.975+0000 7f3fa5abe640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fa0073de0 con 0x7f3fa0142ef0 2024-08-19T13:19:06.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.975+0000 7f3fa5abe640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fa006e280 con 0x7f3fa00a94c0 2024-08-19T13:19:06.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.975+0000 7f3fa5abe640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f3fa0143be0 con 0x7f3fa01437d0 2024-08-19T13:19:06.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.975+0000 7f3f9f7fe640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f3fa0142ef0 0x7f3fa013edc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:42924/0 (socket says 172.21.15.103:42924) 2024-08-19T13:19:06.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.975+0000 7f3f9f7fe640 1 -- 172.21.15.103:0/571439645 learned_addr learned my addr 172.21.15.103:0/571439645 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:19:06.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.975+0000 7f3f9ffff640 1 --2- 172.21.15.103:0/571439645 >> v2:172.21.15.103:3300/0 conn(0x7f3fa01437d0 0x7f3fa014b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:06.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.975+0000 7f3f9ffff640 1 -- 172.21.15.103:0/571439645 >> v1:172.21.15.122:6789/0 conn(0x7f3fa00a94c0 legacy=0x7f3fa0071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:06.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.975+0000 7f3f9ffff640 1 -- 172.21.15.103:0/571439645 >> v1:172.21.15.103:6789/0 conn(0x7f3fa0142ef0 legacy=0x7f3fa013edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:06.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.975+0000 7f3f9ffff640 1 -- 172.21.15.103:0/571439645 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa01432d0 con 0x7f3fa01437d0 2024-08-19T13:19:06.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.976+0000 7f3f9ffff640 1 --2- 172.21.15.103:0/571439645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa01437d0 0x7f3fa014b020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009d90 tx=0x7f3f9802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaeb859aed76b2e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:06.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.976+0000 7f3f9e7fc640 1 -- 172.21.15.103:0/571439645 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98031600 con 0x7f3fa01437d0 2024-08-19T13:19:06.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.976+0000 7f3f9e7fc640 1 -- 172.21.15.103:0/571439645 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f980317a0 con 0x7f3fa01437d0 2024-08-19T13:19:06.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.976+0000 7f3f9e7fc640 1 -- 172.21.15.103:0/571439645 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98031b70 con 0x7f3fa01437d0 2024-08-19T13:19:06.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.977+0000 7f3fa5abe640 1 -- 172.21.15.103:0/571439645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa01437d0 msgr2=0x7f3fa014b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:06.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.977+0000 7f3fa5abe640 1 --2- 172.21.15.103:0/571439645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa01437d0 0x7f3fa014b020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009d90 tx=0x7f3f9802a5d0 comp rx=0 tx=0).stop 2024-08-19T13:19:06.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.977+0000 7f3fa5abe640 1 -- 172.21.15.103:0/571439645 shutdown_connections 2024-08-19T13:19:06.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.977+0000 7f3fa5abe640 1 --2- 172.21.15.103:0/571439645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa01437d0 0x7f3fa014b020 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:06.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.977+0000 7f3fa5abe640 1 -- 172.21.15.103:0/571439645 >> 172.21.15.103:0/571439645 conn(0x7f3fa00b1f70 msgr2=0x7f3fa00b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:06.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.977+0000 7f3fa5abe640 1 -- 172.21.15.103:0/571439645 shutdown_connections 2024-08-19T13:19:06.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.977+0000 7f3fa5abe640 1 -- 172.21.15.103:0/571439645 wait complete. 2024-08-19T13:19:06.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.978+0000 7f3fa5abe640 1 Processor -- start 2024-08-19T13:19:06.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.978+0000 7f3fa5abe640 1 -- start start 2024-08-19T13:19:06.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3fa5abe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3fa0142ef0 0x7f3fa01d9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:06.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3fa5abe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fa01437d0 0x7f3fa00afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:06.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3fa5abe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa00ace80 0x7f3fa00b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:06.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3fa5abe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa0071790 con 0x7f3fa00ace80 2024-08-19T13:19:06.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3fa5abe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa013f4d0 con 0x7f3fa01437d0 2024-08-19T13:19:06.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3fa5abe640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa0071910 con 0x7f3fa0142ef0 2024-08-19T13:19:06.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f9f7fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3fa0142ef0 0x7f3fa01d9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:06.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f9ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa00ace80 0x7f3fa00b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:06.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f9f7fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3fa0142ef0 0x7f3fa01d9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41492/0 (socket says 172.21.15.103:41492) 2024-08-19T13:19:06.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f9f7fe640 1 -- 172.21.15.103:0/3262429653 learned_addr learned my addr 172.21.15.103:0/3262429653 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:06.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f9ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa00ace80 0x7f3fa00b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52578/0 (socket says 172.21.15.103:52578) 2024-08-19T13:19:06.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f9effd640 1 --2- 172.21.15.103:0/3262429653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fa01437d0 0x7f3fa00afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:06.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f9f7fe640 1 -- 172.21.15.103:0/3262429653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fa01437d0 msgr2=0x7f3fa00afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:06.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f9f7fe640 1 --2- 172.21.15.103:0/3262429653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fa01437d0 0x7f3fa00afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:06.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f9f7fe640 1 -- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa00ace80 msgr2=0x7f3fa00b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:06.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f9f7fe640 1 --2- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa00ace80 0x7f3fa00b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:06.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f9f7fe640 1 -- 172.21.15.103:0/3262429653 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa00b01a0 con 0x7f3fa0142ef0 2024-08-19T13:19:06.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f9ffff640 1 --2- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa00ace80 0x7f3fa00b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:06.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f9f7fe640 1 --2- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3fa0142ef0 0x7f3fa01d9470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000c4d0 tx=0x7f3f9000c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:06.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f87fff640 1 -- 172.21.15.103:0/3262429653 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90017070 con 0x7f3fa0142ef0 2024-08-19T13:19:06.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f87fff640 1 -- 172.21.15.103:0/3262429653 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f90005ce0 con 0x7f3fa0142ef0 2024-08-19T13:19:06.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3f87fff640 1 -- 172.21.15.103:0/3262429653 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90007270 con 0x7f3fa0142ef0 2024-08-19T13:19:06.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.979+0000 7f3fa5abe640 1 -- 172.21.15.103:0/3262429653 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa01d6d20 con 0x7f3fa0142ef0 2024-08-19T13:19:06.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.980+0000 7f3fa5abe640 1 -- 172.21.15.103:0/3262429653 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa01d9f30 con 0x7f3fa0142ef0 2024-08-19T13:19:06.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.980+0000 7f3fa5abe640 1 -- 172.21.15.103:0/3262429653 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3fa00ad450 con 0x7f3fa0142ef0 2024-08-19T13:19:06.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.981+0000 7f3f87fff640 1 -- 172.21.15.103:0/3262429653 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f9001a070 con 0x7f3fa0142ef0 2024-08-19T13:19:06.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.981+0000 7f3f87fff640 1 --2- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f3f8c06be60 0x7f3f8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:06.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.981+0000 7f3f87fff640 1 -- 172.21.15.103:0/3262429653 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7f3f9008a010 con 0x7f3fa0142ef0 2024-08-19T13:19:06.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.981+0000 7f3f9effd640 1 --2- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f3f8c06be60 0x7f3f8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:06.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.981+0000 7f3f87fff640 1 --2- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:6808/1798955708,v1:172.21.15.103:6809/1798955708] conn(0x7f3f8c073020 0x7f3f8c075440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:06.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.981+0000 7f3f87fff640 1 -- 172.21.15.103:0/3262429653 --> [v2:172.21.15.103:6808/1798955708,v1:172.21.15.103:6809/1798955708] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f90014ec0 con 0x7f3f8c073020 2024-08-19T13:19:06.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.981+0000 7f3f87fff640 1 -- 172.21.15.103:0/3262429653 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3f90091050 con 0x7f3fa0142ef0 2024-08-19T13:19:06.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.981+0000 7f3f9ffff640 1 --2- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:6808/1798955708,v1:172.21.15.103:6809/1798955708] conn(0x7f3f8c073020 0x7f3f8c075440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:06.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.982+0000 7f3f9effd640 1 --2- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f3f8c06be60 0x7f3f8c06e320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3fa013f8b0 tx=0x7f3f94009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:06.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.982+0000 7f3f9ffff640 1 --2- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:6808/1798955708,v1:172.21.15.103:6809/1798955708] conn(0x7f3f8c073020 0x7f3f8c075440 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:06.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.982+0000 7f3f87fff640 1 -- 172.21.15.103:0/3262429653 <== osd.0 v2:172.21.15.103:6808/1798955708 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f90014ec0 con 0x7f3f8c073020 2024-08-19T13:19:06.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.995+0000 7f3fa5abe640 1 -- 172.21.15.103:0/3262429653 --> [v2:172.21.15.103:6808/1798955708,v1:172.21.15.103:6809/1798955708] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3fa00affb0 con 0x7f3f8c073020 2024-08-19T13:19:06.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.995+0000 7f3f87fff640 1 -- 172.21.15.103:0/3262429653 <== osd.0 v2:172.21.15.103:6808/1798955708 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3fa00affb0 con 0x7f3f8c073020 2024-08-19T13:19:06.997 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T13:19:06.997 INFO:teuthology.orchestra.run.smithi103.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-19T13:19:06.997 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T13:19:06.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.995+0000 7f3fa5abe640 1 -- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:6808/1798955708,v1:172.21.15.103:6809/1798955708] conn(0x7f3f8c073020 msgr2=0x7f3f8c075440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:06.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.995+0000 7f3fa5abe640 1 --2- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:6808/1798955708,v1:172.21.15.103:6809/1798955708] conn(0x7f3f8c073020 0x7f3f8c075440 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:06.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.995+0000 7f3fa5abe640 1 -- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f3f8c06be60 msgr2=0x7f3f8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:06.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.995+0000 7f3fa5abe640 1 --2- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f3f8c06be60 0x7f3f8c06e320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3fa013f8b0 tx=0x7f3f94009040 comp rx=0 tx=0).stop 2024-08-19T13:19:07.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.995+0000 7f3fa5abe640 1 -- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3fa0142ef0 msgr2=0x7f3fa01d9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.995+0000 7f3fa5abe640 1 --2- 172.21.15.103:0/3262429653 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3fa0142ef0 0x7f3fa01d9470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000c4d0 tx=0x7f3f9000c990 comp rx=0 tx=0).stop 2024-08-19T13:19:07.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.995+0000 7f3f9ffff640 1 -- 172.21.15.103:0/3262429653 reap_dead start 2024-08-19T13:19:07.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.995+0000 7f3fa5abe640 1 -- 172.21.15.103:0/3262429653 shutdown_connections 2024-08-19T13:19:07.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.995+0000 7f3fa5abe640 1 -- 172.21.15.103:0/3262429653 >> 172.21.15.103:0/3262429653 conn(0x7f3fa00b1f70 msgr2=0x7f3fa006c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:07.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.996+0000 7f3fa5abe640 1 -- 172.21.15.103:0/3262429653 shutdown_connections 2024-08-19T13:19:07.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:06.996+0000 7f3fa5abe640 1 -- 172.21.15.103:0/3262429653 wait complete. 2024-08-19T13:19:07.007 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-08-19T13:19:07.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.107+0000 7fef99d3d640 1 Processor -- start 2024-08-19T13:19:07.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.107+0000 7fef99d3d640 1 -- start start 2024-08-19T13:19:07.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.107+0000 7fef99d3d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fef940723c0 0x7fef94070140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.107+0000 7fef99d3d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef94139c10 0x7fef94140810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.107+0000 7fef99d3d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef940b2030 con 0x7fef941403a0 2024-08-19T13:19:07.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.107+0000 7fef99d3d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fef940a96e0 con 0x7fef940723c0 2024-08-19T13:19:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.107+0000 7fef99d3d640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fef940a9860 con 0x7fef94139c10 2024-08-19T13:19:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.108+0000 7fef93fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef94139c10 0x7fef94140810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.108+0000 7fef93fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef94139c10 0x7fef94140810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41494/0 (socket says 172.21.15.103:41494) 2024-08-19T13:19:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.108+0000 7fef93fff640 1 -- 172.21.15.103:0/3652801940 learned_addr learned my addr 172.21.15.103:0/3652801940 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.108+0000 7fef937fe640 1 --2- 172.21.15.103:0/3652801940 >> v2:172.21.15.122:3300/0 conn(0x7fef940723c0 0x7fef94070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.108+0000 7fef93fff640 1 -- 172.21.15.103:0/3652801940 >> v2:172.21.15.122:3300/0 conn(0x7fef940723c0 msgr2=0x7fef94070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.108+0000 7fef93fff640 1 --2- 172.21.15.103:0/3652801940 >> v2:172.21.15.122:3300/0 conn(0x7fef940723c0 0x7fef94070140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.108+0000 7fef93fff640 1 -- 172.21.15.103:0/3652801940 >> v1:172.21.15.122:6789/0 conn(0x7fef941403a0 legacy=0x7fef94139500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.108+0000 7fef93fff640 1 -- 172.21.15.103:0/3652801940 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef940732f0 con 0x7fef94139c10 2024-08-19T13:19:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.109+0000 7fef93fff640 1 --2- 172.21.15.103:0/3652801940 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef94139c10 0x7fef94140810 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009e50 tx=0x7fef8c02a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=385c510e882a96cd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.109+0000 7fef927fc640 1 -- 172.21.15.103:0/3652801940 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c036070 con 0x7fef94139c10 2024-08-19T13:19:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.109+0000 7fef927fc640 1 -- 172.21.15.103:0/3652801940 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8c031600 con 0x7fef94139c10 2024-08-19T13:19:07.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.109+0000 7fef927fc640 1 -- 172.21.15.103:0/3652801940 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0319d0 con 0x7fef94139c10 2024-08-19T13:19:07.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.109+0000 7fef99d3d640 1 -- 172.21.15.103:0/3652801940 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef94139c10 msgr2=0x7fef94140810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.109+0000 7fef99d3d640 1 --2- 172.21.15.103:0/3652801940 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef94139c10 0x7fef94140810 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009e50 tx=0x7fef8c02a630 comp rx=0 tx=0).stop 2024-08-19T13:19:07.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.109+0000 7fef99d3d640 1 -- 172.21.15.103:0/3652801940 shutdown_connections 2024-08-19T13:19:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.109+0000 7fef99d3d640 1 --2- 172.21.15.103:0/3652801940 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef94139c10 0x7fef94140810 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.109+0000 7fef99d3d640 1 --2- 172.21.15.103:0/3652801940 >> v2:172.21.15.122:3300/0 conn(0x7fef940723c0 0x7fef94070140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.109+0000 7fef99d3d640 1 -- 172.21.15.103:0/3652801940 >> 172.21.15.103:0/3652801940 conn(0x7fef940681a0 msgr2=0x7fef940b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.109+0000 7fef99d3d640 1 -- 172.21.15.103:0/3652801940 shutdown_connections 2024-08-19T13:19:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.109+0000 7fef99d3d640 1 -- 172.21.15.103:0/3652801940 wait complete. 2024-08-19T13:19:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.110+0000 7fef99d3d640 1 Processor -- start 2024-08-19T13:19:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.110+0000 7fef99d3d640 1 -- start start 2024-08-19T13:19:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef99d3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef940723c0 0x7fef94143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef99d3d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef94139c10 0x7fef9413a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef99d3d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef941403a0 0x7fef9413c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef99d3d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fef94072920 con 0x7fef940723c0 2024-08-19T13:19:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef99d3d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fef940727a0 con 0x7fef941403a0 2024-08-19T13:19:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef99d3d640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fef9414a3c0 con 0x7fef94139c10 2024-08-19T13:19:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef92ffd640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef94139c10 0x7fef9413a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef937fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef940723c0 0x7fef94143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef937fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef940723c0 0x7fef94143860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52582/0 (socket says 172.21.15.103:52582) 2024-08-19T13:19:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef92ffd640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef94139c10 0x7fef9413a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41510/0 (socket says 172.21.15.103:41510) 2024-08-19T13:19:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef937fe640 1 -- 172.21.15.103:0/595135989 learned_addr learned my addr 172.21.15.103:0/595135989 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef93fff640 1 --2- 172.21.15.103:0/595135989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef941403a0 0x7fef9413c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef937fe640 1 -- 172.21.15.103:0/595135989 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef94139c10 msgr2=0x7fef9413a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef937fe640 1 --2- 172.21.15.103:0/595135989 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef94139c10 0x7fef9413a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef937fe640 1 -- 172.21.15.103:0/595135989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef941403a0 msgr2=0x7fef9413c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef937fe640 1 --2- 172.21.15.103:0/595135989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef941403a0 0x7fef9413c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef937fe640 1 -- 172.21.15.103:0/595135989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef941332f0 con 0x7fef940723c0 2024-08-19T13:19:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.111+0000 7fef92ffd640 1 --2- 172.21.15.103:0/595135989 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef94139c10 0x7fef9413a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.112+0000 7fef937fe640 1 --2- 172.21.15.103:0/595135989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef940723c0 0x7fef94143860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fef8400d830 tx=0x7fef8400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.112+0000 7fef7bfff640 1 -- 172.21.15.103:0/595135989 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8400b660 con 0x7fef940723c0 2024-08-19T13:19:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.112+0000 7fef7bfff640 1 -- 172.21.15.103:0/595135989 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef84019070 con 0x7fef940723c0 2024-08-19T13:19:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.112+0000 7fef7bfff640 1 -- 172.21.15.103:0/595135989 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8400ba20 con 0x7fef940723c0 2024-08-19T13:19:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.112+0000 7fef99d3d640 1 -- 172.21.15.103:0/595135989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9413a7d0 con 0x7fef940723c0 2024-08-19T13:19:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.112+0000 7fef99d3d640 1 -- 172.21.15.103:0/595135989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9413cb30 con 0x7fef940723c0 2024-08-19T13:19:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.113+0000 7fef99d3d640 1 -- 172.21.15.103:0/595135989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef940a94c0 con 0x7fef940723c0 2024-08-19T13:19:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.113+0000 7fef7bfff640 1 -- 172.21.15.103:0/595135989 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef8401e460 con 0x7fef940723c0 2024-08-19T13:19:07.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.114+0000 7fef7bfff640 1 --2- 172.21.15.103:0/595135989 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fef8006be30 0x7fef8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.114+0000 7fef7bfff640 1 -- 172.21.15.103:0/595135989 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7fef8408b340 con 0x7fef940723c0 2024-08-19T13:19:07.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.114+0000 7fef92ffd640 1 --2- 172.21.15.103:0/595135989 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fef8006be30 0x7fef8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.114+0000 7fef92ffd640 1 --2- 172.21.15.103:0/595135989 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fef8006be30 0x7fef8006e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fef9413b1a0 tx=0x7fef88009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.118+0000 7fef7bfff640 1 -- 172.21.15.103:0/595135989 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fef840596e0 con 0x7fef940723c0 2024-08-19T13:19:07.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.285+0000 7fef99d3d640 1 -- 172.21.15.103:0/595135989 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fef94140f30 con 0x7fef8006be30 2024-08-19T13:19:07.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.285+0000 7fef7bfff640 1 -- 172.21.15.103:0/595135989 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fef88009e10 con 0x7fef8006be30 2024-08-19T13:19:07.286 INFO:teuthology.orchestra.run.smithi103.stderr:instructed osd(s) 0 to deep-scrub 2024-08-19T13:19:07.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.287+0000 7fef99d3d640 1 -- 172.21.15.103:0/595135989 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fef8006be30 msgr2=0x7fef8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.287+0000 7fef99d3d640 1 --2- 172.21.15.103:0/595135989 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fef8006be30 0x7fef8006e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fef9413b1a0 tx=0x7fef88009060 comp rx=0 tx=0).stop 2024-08-19T13:19:07.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.287+0000 7fef99d3d640 1 -- 172.21.15.103:0/595135989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef940723c0 msgr2=0x7fef94143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.287+0000 7fef99d3d640 1 --2- 172.21.15.103:0/595135989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef940723c0 0x7fef94143860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fef8400d830 tx=0x7fef8400dcf0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.287+0000 7fef99d3d640 1 -- 172.21.15.103:0/595135989 shutdown_connections 2024-08-19T13:19:07.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.287+0000 7fef99d3d640 1 --2- 172.21.15.103:0/595135989 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fef8006be30 0x7fef8006e2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.287+0000 7fef99d3d640 1 --2- 172.21.15.103:0/595135989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef941403a0 0x7fef9413c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.287+0000 7fef99d3d640 1 --2- 172.21.15.103:0/595135989 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef94139c10 0x7fef9413a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.287+0000 7fef99d3d640 1 --2- 172.21.15.103:0/595135989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef940723c0 0x7fef94143860 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.287+0000 7fef99d3d640 1 -- 172.21.15.103:0/595135989 >> 172.21.15.103:0/595135989 conn(0x7fef940681a0 msgr2=0x7fef940709a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:07.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.287+0000 7fef99d3d640 1 -- 172.21.15.103:0/595135989 shutdown_connections 2024-08-19T13:19:07.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.287+0000 7fef99d3d640 1 -- 172.21.15.103:0/595135989 wait complete. 2024-08-19T13:19:07.299 INFO:tasks.ceph:Scrubbing osd.1 2024-08-19T13:19:07.299 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-08-19T13:19:07.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.400+0000 7efd641df640 1 Processor -- start 2024-08-19T13:19:07.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.400+0000 7efd641df640 1 -- start start 2024-08-19T13:19:07.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd641df640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efd5c071cc0 0x7efd5c1436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd641df640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7efd5c0720a0 0x7efd5c142b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd641df640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7efd5c06ffa0 0x7efd5c070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd641df640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7efd5c0724c0 con 0x7efd5c0720a0 2024-08-19T13:19:07.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd641df640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7efd5c0adbb0 con 0x7efd5c06ffa0 2024-08-19T13:19:07.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd641df640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efd5c1431d0 con 0x7efd5c071cc0 2024-08-19T13:19:07.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd61f7b640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efd5c071cc0 0x7efd5c1436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd61f7b640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efd5c071cc0 0x7efd5c1436f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41518/0 (socket says 172.21.15.103:41518) 2024-08-19T13:19:07.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd61f7b640 1 -- 172.21.15.103:0/3630581851 learned_addr learned my addr 172.21.15.103:0/3630581851 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:07.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd6177a640 1 --2- 172.21.15.103:0/3630581851 >> v2:172.21.15.103:3300/0 conn(0x7efd5c0720a0 0x7efd5c142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd6277c640 1 --2- 172.21.15.103:0/3630581851 >> v2:172.21.15.122:3300/0 conn(0x7efd5c06ffa0 0x7efd5c070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd6177a640 1 -- 172.21.15.103:0/3630581851 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efd5c071cc0 msgr2=0x7efd5c1436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:07.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd6177a640 1 --2- 172.21.15.103:0/3630581851 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efd5c071cc0 0x7efd5c1436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd6177a640 1 -- 172.21.15.103:0/3630581851 >> v2:172.21.15.122:3300/0 conn(0x7efd5c06ffa0 msgr2=0x7efd5c070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:07.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd6177a640 1 --2- 172.21.15.103:0/3630581851 >> v2:172.21.15.122:3300/0 conn(0x7efd5c06ffa0 0x7efd5c070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd6177a640 1 -- 172.21.15.103:0/3630581851 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c13fb00 con 0x7efd5c0720a0 2024-08-19T13:19:07.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd61f7b640 1 --2- 172.21.15.103:0/3630581851 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efd5c071cc0 0x7efd5c1436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:07.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.401+0000 7efd6277c640 1 --2- 172.21.15.103:0/3630581851 >> v2:172.21.15.122:3300/0 conn(0x7efd5c06ffa0 0x7efd5c070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T13:19:07.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.402+0000 7efd6177a640 1 --2- 172.21.15.103:0/3630581851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c0720a0 0x7efd5c142b00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efd5c147160 tx=0x7efd4c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab108602e78ec26 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.402+0000 7efd60f79640 1 -- 172.21.15.103:0/3630581851 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c037070 con 0x7efd5c0720a0 2024-08-19T13:19:07.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.402+0000 7efd60f79640 1 -- 172.21.15.103:0/3630581851 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd4c02ae50 con 0x7efd5c0720a0 2024-08-19T13:19:07.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.402+0000 7efd60f79640 1 -- 172.21.15.103:0/3630581851 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c032660 con 0x7efd5c0720a0 2024-08-19T13:19:07.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.403+0000 7efd641df640 1 -- 172.21.15.103:0/3630581851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c0720a0 msgr2=0x7efd5c142b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.403+0000 7efd641df640 1 --2- 172.21.15.103:0/3630581851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c0720a0 0x7efd5c142b00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efd5c147160 tx=0x7efd4c02a100 comp rx=0 tx=0).stop 2024-08-19T13:19:07.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.403+0000 7efd641df640 1 -- 172.21.15.103:0/3630581851 shutdown_connections 2024-08-19T13:19:07.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.403+0000 7efd641df640 1 --2- 172.21.15.103:0/3630581851 >> v2:172.21.15.122:3300/0 conn(0x7efd5c06ffa0 0x7efd5c070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.403+0000 7efd641df640 1 --2- 172.21.15.103:0/3630581851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c0720a0 0x7efd5c142b00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.403+0000 7efd641df640 1 --2- 172.21.15.103:0/3630581851 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efd5c071cc0 0x7efd5c1436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.403+0000 7efd641df640 1 -- 172.21.15.103:0/3630581851 >> 172.21.15.103:0/3630581851 conn(0x7efd5c0b5370 msgr2=0x7efd5c0b5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:07.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.403+0000 7efd641df640 1 -- 172.21.15.103:0/3630581851 shutdown_connections 2024-08-19T13:19:07.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.403+0000 7efd641df640 1 -- 172.21.15.103:0/3630581851 wait complete. 2024-08-19T13:19:07.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.404+0000 7efd641df640 1 Processor -- start 2024-08-19T13:19:07.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.404+0000 7efd641df640 1 -- start start 2024-08-19T13:19:07.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.404+0000 7efd641df640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efd5c06ffa0 0x7efd5c0ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.404+0000 7efd641df640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd5c071cc0 0x7efd5c0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.404+0000 7efd641df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c0720a0 0x7efd5c1db860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.404+0000 7efd641df640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c140210 con 0x7efd5c0720a0 2024-08-19T13:19:07.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.404+0000 7efd641df640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c140090 con 0x7efd5c071cc0 2024-08-19T13:19:07.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.404+0000 7efd641df640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efd5c140390 con 0x7efd5c06ffa0 2024-08-19T13:19:07.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd61f7b640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efd5c06ffa0 0x7efd5c0ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd6277c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c0720a0 0x7efd5c1db860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd61f7b640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efd5c06ffa0 0x7efd5c0ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41534/0 (socket says 172.21.15.103:41534) 2024-08-19T13:19:07.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd61f7b640 1 -- 172.21.15.103:0/553779834 learned_addr learned my addr 172.21.15.103:0/553779834 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:07.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd6177a640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd5c071cc0 0x7efd5c0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd61f7b640 1 -- 172.21.15.103:0/553779834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd5c071cc0 msgr2=0x7efd5c0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd61f7b640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd5c071cc0 0x7efd5c0aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd61f7b640 1 -- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c0720a0 msgr2=0x7efd5c1db860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd61f7b640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c0720a0 0x7efd5c1db860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd61f7b640 1 -- 172.21.15.103:0/553779834 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c0b0600 con 0x7efd5c06ffa0 2024-08-19T13:19:07.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd6277c640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c0720a0 0x7efd5c1db860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:07.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd6177a640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd5c071cc0 0x7efd5c0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T13:19:07.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd61f7b640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efd5c06ffa0 0x7efd5c0ae340 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd5800e9d0 tx=0x7efd5800ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd527fc640 1 -- 172.21.15.103:0/553779834 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58017070 con 0x7efd5c06ffa0 2024-08-19T13:19:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd527fc640 1 -- 172.21.15.103:0/553779834 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5800ca60 con 0x7efd5c06ffa0 2024-08-19T13:19:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd641df640 1 -- 172.21.15.103:0/553779834 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5c1db340 con 0x7efd5c06ffa0 2024-08-19T13:19:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd527fc640 1 -- 172.21.15.103:0/553779834 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5800ce10 con 0x7efd5c06ffa0 2024-08-19T13:19:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.405+0000 7efd641df640 1 -- 172.21.15.103:0/553779834 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5c144400 con 0x7efd5c06ffa0 2024-08-19T13:19:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.406+0000 7efd641df640 1 -- 172.21.15.103:0/553779834 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd5c1436f0 con 0x7efd5c06ffa0 2024-08-19T13:19:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.407+0000 7efd527fc640 1 -- 172.21.15.103:0/553779834 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd5801a070 con 0x7efd5c06ffa0 2024-08-19T13:19:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.407+0000 7efd527fc640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7efd4006be30 0x7efd4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.407+0000 7efd527fc640 1 -- 172.21.15.103:0/553779834 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7efd5808af90 con 0x7efd5c06ffa0 2024-08-19T13:19:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.407+0000 7efd6177a640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7efd4006be30 0x7efd4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.407+0000 7efd527fc640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:6824/855529591,v1:172.21.15.103:6825/855529591] conn(0x7efd40072ff0 0x7efd40075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.407+0000 7efd527fc640 1 -- 172.21.15.103:0/553779834 --> [v2:172.21.15.103:6824/855529591,v1:172.21.15.103:6825/855529591] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd58016ec0 con 0x7efd40072ff0 2024-08-19T13:19:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.407+0000 7efd527fc640 1 -- 172.21.15.103:0/553779834 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efd58092050 con 0x7efd5c06ffa0 2024-08-19T13:19:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.407+0000 7efd6277c640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:6824/855529591,v1:172.21.15.103:6825/855529591] conn(0x7efd40072ff0 0x7efd40075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.407+0000 7efd6177a640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7efd4006be30 0x7efd4006e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd4c034000 tx=0x7efd4c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.408+0000 7efd6277c640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:6824/855529591,v1:172.21.15.103:6825/855529591] conn(0x7efd40072ff0 0x7efd40075410 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.408+0000 7efd527fc640 1 -- 172.21.15.103:0/553779834 <== osd.1 v2:172.21.15.103:6824/855529591 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd58016ec0 con 0x7efd40072ff0 2024-08-19T13:19:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.420+0000 7efd641df640 1 -- 172.21.15.103:0/553779834 --> [v2:172.21.15.103:6824/855529591,v1:172.21.15.103:6825/855529591] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efd5c1dbff0 con 0x7efd40072ff0 2024-08-19T13:19:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.421+0000 7efd527fc640 1 -- 172.21.15.103:0/553779834 <== osd.1 v2:172.21.15.103:6824/855529591 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efd5c1dbff0 con 0x7efd40072ff0 2024-08-19T13:19:07.422 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T13:19:07.422 INFO:teuthology.orchestra.run.smithi103.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-19T13:19:07.422 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T13:19:07.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.421+0000 7efd641df640 1 -- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:6824/855529591,v1:172.21.15.103:6825/855529591] conn(0x7efd40072ff0 msgr2=0x7efd40075410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.421+0000 7efd641df640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:6824/855529591,v1:172.21.15.103:6825/855529591] conn(0x7efd40072ff0 0x7efd40075410 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.421+0000 7efd641df640 1 -- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7efd4006be30 msgr2=0x7efd4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.421+0000 7efd641df640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7efd4006be30 0x7efd4006e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd4c034000 tx=0x7efd4c02e000 comp rx=0 tx=0).stop 2024-08-19T13:19:07.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.421+0000 7efd641df640 1 -- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efd5c06ffa0 msgr2=0x7efd5c0ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.421+0000 7efd641df640 1 --2- 172.21.15.103:0/553779834 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efd5c06ffa0 0x7efd5c0ae340 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd5800e9d0 tx=0x7efd5800ee90 comp rx=0 tx=0).stop 2024-08-19T13:19:07.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.421+0000 7efd6277c640 1 -- 172.21.15.103:0/553779834 reap_dead start 2024-08-19T13:19:07.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.421+0000 7efd641df640 1 -- 172.21.15.103:0/553779834 shutdown_connections 2024-08-19T13:19:07.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.421+0000 7efd641df640 1 -- 172.21.15.103:0/553779834 >> 172.21.15.103:0/553779834 conn(0x7efd5c0b5370 msgr2=0x7efd5c13bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:07.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.421+0000 7efd641df640 1 -- 172.21.15.103:0/553779834 shutdown_connections 2024-08-19T13:19:07.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.421+0000 7efd641df640 1 -- 172.21.15.103:0/553779834 wait complete. 2024-08-19T13:19:07.432 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-08-19T13:19:07.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.530+0000 7fee730cb640 1 Processor -- start 2024-08-19T13:19:07.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.530+0000 7fee730cb640 1 -- start start 2024-08-19T13:19:07.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee730cb640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fee6c06ff20 0x7fee6c142170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee730cb640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c1426b0 0x7fee6c146870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee730cb640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee6c070300 con 0x7fee6c06f760 2024-08-19T13:19:07.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee730cb640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fee6c073a70 con 0x7fee6c06ff20 2024-08-19T13:19:07.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee730cb640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fee6c146db0 con 0x7fee6c1426b0 2024-08-19T13:19:07.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee71668640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c1426b0 0x7fee6c146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee70e67640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fee6c06ff20 0x7fee6c142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee70e67640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fee6c06ff20 0x7fee6c142170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52612/0 (socket says 172.21.15.103:52612) 2024-08-19T13:19:07.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee71668640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c1426b0 0x7fee6c146870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41546/0 (socket says 172.21.15.103:41546) 2024-08-19T13:19:07.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee70e67640 1 -- 172.21.15.103:0/2651813534 learned_addr learned my addr 172.21.15.103:0/2651813534 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:07.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee70e67640 1 -- 172.21.15.103:0/2651813534 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c1426b0 msgr2=0x7fee6c146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:07.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee70e67640 1 --2- 172.21.15.103:0/2651813534 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c1426b0 0x7fee6c146870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee70e67640 1 -- 172.21.15.103:0/2651813534 >> v1:172.21.15.103:6789/0 conn(0x7fee6c06f760 legacy=0x7fee6c146160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee70e67640 1 -- 172.21.15.103:0/2651813534 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6c073f00 con 0x7fee6c06ff20 2024-08-19T13:19:07.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.531+0000 7fee71668640 1 --2- 172.21.15.103:0/2651813534 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c1426b0 0x7fee6c146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:07.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.532+0000 7fee70e67640 1 --2- 172.21.15.103:0/2651813534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c06ff20 0x7fee6c142170 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009870 tx=0x7fee5c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d3e3d794d776bb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.532+0000 7fee6b7fe640 1 -- 172.21.15.103:0/2651813534 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c037070 con 0x7fee6c06ff20 2024-08-19T13:19:07.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.532+0000 7fee6b7fe640 1 -- 172.21.15.103:0/2651813534 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee5c032440 con 0x7fee6c06ff20 2024-08-19T13:19:07.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.532+0000 7fee6b7fe640 1 -- 172.21.15.103:0/2651813534 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c032810 con 0x7fee6c06ff20 2024-08-19T13:19:07.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.533+0000 7fee730cb640 1 -- 172.21.15.103:0/2651813534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c06ff20 msgr2=0x7fee6c142170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.533+0000 7fee730cb640 1 --2- 172.21.15.103:0/2651813534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c06ff20 0x7fee6c142170 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009870 tx=0x7fee5c02a1c0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.533+0000 7fee730cb640 1 -- 172.21.15.103:0/2651813534 shutdown_connections 2024-08-19T13:19:07.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.533+0000 7fee730cb640 1 --2- 172.21.15.103:0/2651813534 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c1426b0 0x7fee6c146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.533+0000 7fee730cb640 1 --2- 172.21.15.103:0/2651813534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c06ff20 0x7fee6c142170 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.533+0000 7fee730cb640 1 -- 172.21.15.103:0/2651813534 >> 172.21.15.103:0/2651813534 conn(0x7fee6c0b6bb0 msgr2=0x7fee6c0b6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:07.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.533+0000 7fee730cb640 1 -- 172.21.15.103:0/2651813534 shutdown_connections 2024-08-19T13:19:07.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.533+0000 7fee730cb640 1 -- 172.21.15.103:0/2651813534 wait complete. 2024-08-19T13:19:07.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.534+0000 7fee730cb640 1 Processor -- start 2024-08-19T13:19:07.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.534+0000 7fee730cb640 1 -- start start 2024-08-19T13:19:07.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.534+0000 7fee730cb640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c06f760 0x7fee6c1dba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.534+0000 7fee730cb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee6c06ff20 0x7fee6c1dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.534+0000 7fee730cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c1426b0 0x7fee6c0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.534+0000 7fee730cb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c06f380 con 0x7fee6c1426b0 2024-08-19T13:19:07.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.534+0000 7fee730cb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c0722d0 con 0x7fee6c06ff20 2024-08-19T13:19:07.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.534+0000 7fee730cb640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fee6c06f500 con 0x7fee6c06f760 2024-08-19T13:19:07.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.534+0000 7fee70e67640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c06f760 0x7fee6c1dba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.534+0000 7fee71668640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c1426b0 0x7fee6c0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.534+0000 7fee70e67640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c06f760 0x7fee6c1dba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41550/0 (socket says 172.21.15.103:41550) 2024-08-19T13:19:07.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 -- 172.21.15.103:0/912980921 learned_addr learned my addr 172.21.15.103:0/912980921 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:07.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 -- 172.21.15.103:0/912980921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee6c06ff20 msgr2=0x7fee6c1dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee6bfff640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee6c06ff20 0x7fee6c1dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee6c06ff20 0x7fee6c1dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 -- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c1426b0 msgr2=0x7fee6c0ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c1426b0 0x7fee6c0ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 -- 172.21.15.103:0/912980921 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6c1dc4c0 con 0x7fee6c06f760 2024-08-19T13:19:07.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee71668640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c1426b0 0x7fee6c0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:07.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c06f760 0x7fee6c1dba40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000c00 tx=0x7fee5c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 -- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c06f760 msgr2=0x7fee6c1dba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T13:19:07.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 -- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c06f760 msgr2=0x7fee6c1dba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T13:19:07.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c06f760 0x7fee6c1dba40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000c00 tx=0x7fee5c0046c0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:19:07.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c06f760 0x7fee6c1dba40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000c00 tx=0x7fee5c0046c0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee697fa640 1 -- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c06f760 msgr2=0x7fee6c1dba40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T13:19:07.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee697fa640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee6c06f760 0x7fee6c1dba40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee697fa640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c1426b0 0x7fee580054b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee697fa640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee58005ef0 0x7fee58008380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee697fa640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee6c06ff20 0x7fee5800af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee697fa640 1 -- 172.21.15.103:0/912980921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee5800b9a0 con 0x7fee6c1426b0 2024-08-19T13:19:07.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee697fa640 1 -- 172.21.15.103:0/912980921 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fee5800bb50 con 0x7fee6c06ff20 2024-08-19T13:19:07.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee697fa640 1 -- 172.21.15.103:0/912980921 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fee5800bd50 con 0x7fee58005ef0 2024-08-19T13:19:07.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c1426b0 0x7fee580054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee6bfff640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee58005ef0 0x7fee58008380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 -- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee58005ef0 msgr2=0x7fee58008380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee58005ef0 0x7fee58008380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 -- 172.21.15.103:0/912980921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee6c06ff20 msgr2=0x7fee5800af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee71668640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee6c06ff20 0x7fee5800af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee6c06ff20 0x7fee5800af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee70e67640 1 -- 172.21.15.103:0/912980921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5800bf50 con 0x7fee6c1426b0 2024-08-19T13:19:07.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.535+0000 7fee6bfff640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee58005ef0 0x7fee58008380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:07.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.536+0000 7fee70e67640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c1426b0 0x7fee580054b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009870 tx=0x7fee5c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.536+0000 7fee697fa640 1 -- 172.21.15.103:0/912980921 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c037040 con 0x7fee6c1426b0 2024-08-19T13:19:07.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.536+0000 7fee697fa640 1 -- 172.21.15.103:0/912980921 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee5c0309c0 con 0x7fee6c1426b0 2024-08-19T13:19:07.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.536+0000 7fee730cb640 1 -- 172.21.15.103:0/912980921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6c0af320 con 0x7fee6c1426b0 2024-08-19T13:19:07.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.536+0000 7fee697fa640 1 -- 172.21.15.103:0/912980921 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c030db0 con 0x7fee6c1426b0 2024-08-19T13:19:07.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.536+0000 7fee730cb640 1 -- 172.21.15.103:0/912980921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee6c1dbc20 con 0x7fee6c1426b0 2024-08-19T13:19:07.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.537+0000 7fee730cb640 1 -- 172.21.15.103:0/912980921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee6c0b1eb0 con 0x7fee6c1426b0 2024-08-19T13:19:07.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.537+0000 7fee697fa640 1 -- 172.21.15.103:0/912980921 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee5c04f060 con 0x7fee6c1426b0 2024-08-19T13:19:07.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.538+0000 7fee697fa640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fee6c06f760 0x7fee58076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.538+0000 7fee697fa640 1 -- 172.21.15.103:0/912980921 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7fee5c0ac0b0 con 0x7fee6c1426b0 2024-08-19T13:19:07.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.538+0000 7fee6bfff640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fee6c06f760 0x7fee58076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.538+0000 7fee6bfff640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fee6c06f760 0x7fee58076370 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0afbc0 tx=0x7fee60009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.542+0000 7fee697fa640 1 -- 172.21.15.103:0/912980921 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fee6c0b1eb0 con 0x7fee6c1426b0 2024-08-19T13:19:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.709+0000 7fee730cb640 1 -- 172.21.15.103:0/912980921 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fee6c0af550 con 0x7fee6c06f760 2024-08-19T13:19:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.709+0000 7fee697fa640 1 -- 172.21.15.103:0/912980921 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fee6c0af550 con 0x7fee6c06f760 2024-08-19T13:19:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:instructed osd(s) 1 to deep-scrub 2024-08-19T13:19:07.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.712+0000 7fee730cb640 1 -- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fee6c06f760 msgr2=0x7fee58076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.712+0000 7fee730cb640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fee6c06f760 0x7fee58076370 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0afbc0 tx=0x7fee60009040 comp rx=0 tx=0).stop 2024-08-19T13:19:07.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.712+0000 7fee730cb640 1 -- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c1426b0 msgr2=0x7fee580054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.712+0000 7fee730cb640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c1426b0 0x7fee580054b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009870 tx=0x7fee5c02e000 comp rx=0 tx=0).stop 2024-08-19T13:19:07.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.712+0000 7fee730cb640 1 -- 172.21.15.103:0/912980921 shutdown_connections 2024-08-19T13:19:07.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.712+0000 7fee730cb640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fee6c06f760 0x7fee58076370 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.712+0000 7fee730cb640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee6c06ff20 0x7fee5800af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.712+0000 7fee730cb640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee58005ef0 0x7fee58008380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.712+0000 7fee730cb640 1 --2- 172.21.15.103:0/912980921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c1426b0 0x7fee580054b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.712+0000 7fee730cb640 1 -- 172.21.15.103:0/912980921 >> 172.21.15.103:0/912980921 conn(0x7fee6c0b6bb0 msgr2=0x7fee6c0b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:07.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.712+0000 7fee730cb640 1 -- 172.21.15.103:0/912980921 shutdown_connections 2024-08-19T13:19:07.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.712+0000 7fee730cb640 1 -- 172.21.15.103:0/912980921 wait complete. 2024-08-19T13:19:07.724 INFO:tasks.ceph:Scrubbing osd.2 2024-08-19T13:19:07.724 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-08-19T13:19:07.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.825+0000 7f90524e8640 1 Processor -- start 2024-08-19T13:19:07.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.825+0000 7f90524e8640 1 -- start start 2024-08-19T13:19:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.825+0000 7f90524e8640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f904c0732e0 0x7f904c13fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.826+0000 7f90524e8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f904c13c6a0 0x7f904c140480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.826+0000 7f90524e8640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f904c0b1e70 con 0x7f904c0736c0 2024-08-19T13:19:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.826+0000 7f90524e8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f904c13f960 con 0x7f904c13c6a0 2024-08-19T13:19:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.826+0000 7f90524e8640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f904c13f4c0 con 0x7f904c0732e0 2024-08-19T13:19:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.826+0000 7f904bfff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f904c0732e0 0x7f904c13fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.826+0000 7f904bfff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f904c0732e0 0x7f904c13fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41568/0 (socket says 172.21.15.103:41568) 2024-08-19T13:19:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.826+0000 7f904bfff640 1 -- 172.21.15.103:0/3879110025 learned_addr learned my addr 172.21.15.103:0/3879110025 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.826+0000 7f904bfff640 1 -- 172.21.15.103:0/3879110025 >> v2:172.21.15.122:3300/0 conn(0x7f904c13c6a0 msgr2=0x7f904c140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.826+0000 7f904bfff640 1 --2- 172.21.15.103:0/3879110025 >> v2:172.21.15.122:3300/0 conn(0x7f904c13c6a0 0x7f904c140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.826+0000 7f904bfff640 1 -- 172.21.15.103:0/3879110025 >> v1:172.21.15.122:6789/0 conn(0x7f904c0736c0 legacy=0x7f904c13bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.826+0000 7f904bfff640 1 -- 172.21.15.103:0/3879110025 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c13f640 con 0x7f904c0732e0 2024-08-19T13:19:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.827+0000 7f904bfff640 1 --2- 172.21.15.103:0/3879110025 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f904c0732e0 0x7f904c13fdf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f903c009870 tx=0x7f903c02b320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=597236acccffb75b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.827+0000 7f904affd640 1 -- 172.21.15.103:0/3879110025 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c038070 con 0x7f904c0732e0 2024-08-19T13:19:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.827+0000 7f904affd640 1 -- 172.21.15.103:0/3879110025 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903c033640 con 0x7f904c0732e0 2024-08-19T13:19:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.827+0000 7f904affd640 1 -- 172.21.15.103:0/3879110025 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c033a10 con 0x7f904c0732e0 2024-08-19T13:19:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.827+0000 7f90524e8640 1 -- 172.21.15.103:0/3879110025 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f904c0732e0 msgr2=0x7f904c13fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.827+0000 7f90524e8640 1 --2- 172.21.15.103:0/3879110025 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f904c0732e0 0x7f904c13fdf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f903c009870 tx=0x7f903c02b320 comp rx=0 tx=0).stop 2024-08-19T13:19:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.827+0000 7f90524e8640 1 -- 172.21.15.103:0/3879110025 shutdown_connections 2024-08-19T13:19:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.827+0000 7f90524e8640 1 --2- 172.21.15.103:0/3879110025 >> v2:172.21.15.122:3300/0 conn(0x7f904c13c6a0 0x7f904c140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.827+0000 7f90524e8640 1 --2- 172.21.15.103:0/3879110025 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f904c0732e0 0x7f904c13fdf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.827+0000 7f90524e8640 1 -- 172.21.15.103:0/3879110025 >> 172.21.15.103:0/3879110025 conn(0x7f904c0b0f60 msgr2=0x7f904c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.827+0000 7f90524e8640 1 -- 172.21.15.103:0/3879110025 shutdown_connections 2024-08-19T13:19:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.827+0000 7f90524e8640 1 -- 172.21.15.103:0/3879110025 wait complete. 2024-08-19T13:19:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.828+0000 7f90524e8640 1 Processor -- start 2024-08-19T13:19:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f90524e8640 1 -- start start 2024-08-19T13:19:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f90524e8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f904c0732e0 0x7f904c071750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f90524e8640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f904c0736c0 0x7f904c0ab310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f90524e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f904c13c6a0 0x7f904c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f90524e8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f904c140d50 con 0x7f904c0732e0 2024-08-19T13:19:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f90524e8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f904c140bd0 con 0x7f904c13c6a0 2024-08-19T13:19:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f90524e8640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f904c140ed0 con 0x7f904c0736c0 2024-08-19T13:19:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f904b7fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f904c0736c0 0x7f904c0ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f904bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f904c0732e0 0x7f904c071750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f904b7fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f904c0736c0 0x7f904c0ab310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41580/0 (socket says 172.21.15.103:41580) 2024-08-19T13:19:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f904b7fe640 1 -- 172.21.15.103:0/523758029 learned_addr learned my addr 172.21.15.103:0/523758029 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f904bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f904c0732e0 0x7f904c071750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52636/0 (socket says 172.21.15.103:52636) 2024-08-19T13:19:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f9050a85640 1 --2- 172.21.15.103:0/523758029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f904c13c6a0 0x7f904c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f904b7fe640 1 -- 172.21.15.103:0/523758029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f904c13c6a0 msgr2=0x7f904c0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.829+0000 7f904b7fe640 1 --2- 172.21.15.103:0/523758029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f904c13c6a0 0x7f904c0adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.830+0000 7f904b7fe640 1 -- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f904c0732e0 msgr2=0x7f904c071750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.830+0000 7f904b7fe640 1 --2- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f904c0732e0 0x7f904c071750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.830+0000 7f904b7fe640 1 -- 172.21.15.103:0/523758029 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c0ad4f0 con 0x7f904c0736c0 2024-08-19T13:19:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.830+0000 7f904bfff640 1 --2- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f904c0732e0 0x7f904c071750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.830+0000 7f904b7fe640 1 --2- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f904c0736c0 0x7f904c0ab310 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f904000c400 tx=0x7f904000c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.830+0000 7f9048ff9640 1 -- 172.21.15.103:0/523758029 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040017070 con 0x7f904c0736c0 2024-08-19T13:19:07.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.830+0000 7f9048ff9640 1 -- 172.21.15.103:0/523758029 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90400035f0 con 0x7f904c0736c0 2024-08-19T13:19:07.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.830+0000 7f90524e8640 1 -- 172.21.15.103:0/523758029 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904c0ac250 con 0x7f904c0736c0 2024-08-19T13:19:07.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.830+0000 7f9048ff9640 1 -- 172.21.15.103:0/523758029 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040003a20 con 0x7f904c0736c0 2024-08-19T13:19:07.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.830+0000 7f90524e8640 1 -- 172.21.15.103:0/523758029 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f904c0ab850 con 0x7f904c0736c0 2024-08-19T13:19:07.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.831+0000 7f90524e8640 1 -- 172.21.15.103:0/523758029 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f904c13fdf0 con 0x7f904c0736c0 2024-08-19T13:19:07.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.831+0000 7f9048ff9640 1 -- 172.21.15.103:0/523758029 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f904001a070 con 0x7f904c0736c0 2024-08-19T13:19:07.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.832+0000 7f9048ff9640 1 --2- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f903806be30 0x7f903806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.832+0000 7f9048ff9640 1 -- 172.21.15.103:0/523758029 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7f904008a190 con 0x7f904c0736c0 2024-08-19T13:19:07.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.832+0000 7f904bfff640 1 --2- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f903806be30 0x7f903806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.832+0000 7f9048ff9640 1 --2- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:6800/4004268210,v1:172.21.15.103:6801/4004268210] conn(0x7f9038072ff0 0x7f9038075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.832+0000 7f9048ff9640 1 -- 172.21.15.103:0/523758029 --> [v2:172.21.15.103:6800/4004268210,v1:172.21.15.103:6801/4004268210] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9038075ac0 con 0x7f9038072ff0 2024-08-19T13:19:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.832+0000 7f9048ff9640 1 -- 172.21.15.103:0/523758029 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9040091050 con 0x7f904c0736c0 2024-08-19T13:19:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.832+0000 7f9050a85640 1 --2- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:6800/4004268210,v1:172.21.15.103:6801/4004268210] conn(0x7f9038072ff0 0x7f9038075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.832+0000 7f904bfff640 1 --2- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f903806be30 0x7f903806e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f903c035000 tx=0x7f903c02f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.832+0000 7f9050a85640 1 --2- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:6800/4004268210,v1:172.21.15.103:6801/4004268210] conn(0x7f9038072ff0 0x7f9038075410 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.833+0000 7f9048ff9640 1 -- 172.21.15.103:0/523758029 <== osd.2 v2:172.21.15.103:6800/4004268210 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9038075ac0 con 0x7f9038072ff0 2024-08-19T13:19:07.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.845+0000 7f90524e8640 1 -- 172.21.15.103:0/523758029 --> [v2:172.21.15.103:6800/4004268210,v1:172.21.15.103:6801/4004268210] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f904c0ae3f0 con 0x7f9038072ff0 2024-08-19T13:19:07.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.846+0000 7f9048ff9640 1 -- 172.21.15.103:0/523758029 <== osd.2 v2:172.21.15.103:6800/4004268210 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f904c0ae3f0 con 0x7f9038072ff0 2024-08-19T13:19:07.847 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T13:19:07.847 INFO:teuthology.orchestra.run.smithi103.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-19T13:19:07.847 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T13:19:07.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.846+0000 7f90524e8640 1 -- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:6800/4004268210,v1:172.21.15.103:6801/4004268210] conn(0x7f9038072ff0 msgr2=0x7f9038075410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.846+0000 7f90524e8640 1 --2- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:6800/4004268210,v1:172.21.15.103:6801/4004268210] conn(0x7f9038072ff0 0x7f9038075410 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.846+0000 7f90524e8640 1 -- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f903806be30 msgr2=0x7f903806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.846+0000 7f90524e8640 1 --2- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f903806be30 0x7f903806e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f903c035000 tx=0x7f903c02f000 comp rx=0 tx=0).stop 2024-08-19T13:19:07.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.846+0000 7f90524e8640 1 -- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f904c0736c0 msgr2=0x7f904c0ab310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.846+0000 7f90524e8640 1 --2- 172.21.15.103:0/523758029 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f904c0736c0 0x7f904c0ab310 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f904000c400 tx=0x7f904000c8c0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.846+0000 7f9050a85640 1 -- 172.21.15.103:0/523758029 reap_dead start 2024-08-19T13:19:07.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.846+0000 7f90524e8640 1 -- 172.21.15.103:0/523758029 shutdown_connections 2024-08-19T13:19:07.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.846+0000 7f90524e8640 1 -- 172.21.15.103:0/523758029 >> 172.21.15.103:0/523758029 conn(0x7f904c0b0f60 msgr2=0x7f904c072680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:07.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.846+0000 7f90524e8640 1 -- 172.21.15.103:0/523758029 shutdown_connections 2024-08-19T13:19:07.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.846+0000 7f90524e8640 1 -- 172.21.15.103:0/523758029 wait complete. 2024-08-19T13:19:07.857 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-08-19T13:19:07.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.956+0000 7fc8a77d8640 1 Processor -- start 2024-08-19T13:19:07.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.956+0000 7fc8a77d8640 1 -- start start 2024-08-19T13:19:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.957+0000 7fc8a77d8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc8a006f2e0 0x7fc8a0072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.957+0000 7fc8a77d8640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fc8a013c540 0x7fc8a013c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.957+0000 7fc8a77d8640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8a0072780 con 0x7fc8a006f6c0 2024-08-19T13:19:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.957+0000 7fc8a77d8640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fc8a006ebe0 con 0x7fc8a013c540 2024-08-19T13:19:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.957+0000 7fc8a77d8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc8a006ee30 con 0x7fc8a006f2e0 2024-08-19T13:19:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.957+0000 7fc8a5d75640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fc8a013c540 0x7fc8a013c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.957+0000 7fc8a5d75640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fc8a013c540 0x7fc8a013c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52646/0 (socket says 172.21.15.103:52646) 2024-08-19T13:19:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.957+0000 7fc8a5d75640 1 -- 172.21.15.103:0/702079298 learned_addr learned my addr 172.21.15.103:0/702079298 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.957+0000 7fc8a5574640 1 --2- 172.21.15.103:0/702079298 >> v2:172.21.15.122:3300/0 conn(0x7fc8a006f2e0 0x7fc8a0072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.957+0000 7fc8a5d75640 1 -- 172.21.15.103:0/702079298 >> v2:172.21.15.122:3300/0 conn(0x7fc8a006f2e0 msgr2=0x7fc8a0072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:07.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.957+0000 7fc8a5d75640 1 --2- 172.21.15.103:0/702079298 >> v2:172.21.15.122:3300/0 conn(0x7fc8a006f2e0 0x7fc8a0072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.957+0000 7fc8a5d75640 1 -- 172.21.15.103:0/702079298 >> v1:172.21.15.103:6789/0 conn(0x7fc8a006f6c0 legacy=0x7fc8a01432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.957+0000 7fc8a5d75640 1 -- 172.21.15.103:0/702079298 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a013cef0 con 0x7fc8a013c540 2024-08-19T13:19:07.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.958+0000 7fc8a5d75640 1 --2- 172.21.15.103:0/702079298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a013c540 0x7fc8a013c9b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc8a006ed60 tx=0x7fc89802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409bb42cda36233b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.958+0000 7fc897fff640 1 -- 172.21.15.103:0/702079298 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc898036070 con 0x7fc8a013c540 2024-08-19T13:19:07.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.958+0000 7fc897fff640 1 -- 172.21.15.103:0/702079298 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc898031600 con 0x7fc8a013c540 2024-08-19T13:19:07.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.958+0000 7fc897fff640 1 -- 172.21.15.103:0/702079298 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8980319d0 con 0x7fc8a013c540 2024-08-19T13:19:07.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.959+0000 7fc8a77d8640 1 -- 172.21.15.103:0/702079298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a013c540 msgr2=0x7fc8a013c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.959+0000 7fc8a77d8640 1 --2- 172.21.15.103:0/702079298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a013c540 0x7fc8a013c9b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc8a006ed60 tx=0x7fc89802a6a0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.959+0000 7fc8a77d8640 1 -- 172.21.15.103:0/702079298 shutdown_connections 2024-08-19T13:19:07.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.959+0000 7fc8a77d8640 1 --2- 172.21.15.103:0/702079298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a013c540 0x7fc8a013c9b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.959+0000 7fc8a77d8640 1 --2- 172.21.15.103:0/702079298 >> v2:172.21.15.122:3300/0 conn(0x7fc8a006f2e0 0x7fc8a0072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.959+0000 7fc8a77d8640 1 -- 172.21.15.103:0/702079298 >> 172.21.15.103:0/702079298 conn(0x7fc8a00b3430 msgr2=0x7fc8a00b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:07.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.959+0000 7fc8a77d8640 1 -- 172.21.15.103:0/702079298 shutdown_connections 2024-08-19T13:19:07.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.959+0000 7fc8a77d8640 1 -- 172.21.15.103:0/702079298 wait complete. 2024-08-19T13:19:07.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.960+0000 7fc8a77d8640 1 Processor -- start 2024-08-19T13:19:07.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.960+0000 7fc8a77d8640 1 -- start start 2024-08-19T13:19:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.960+0000 7fc8a77d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a006f2e0 0x7fc8a013db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.960+0000 7fc8a77d8640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8a006f6c0 0x7fc8a0135a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.960+0000 7fc8a77d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8a013c540 0x7fc8a0135f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.960+0000 7fc8a77d8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a006fc00 con 0x7fc8a006f2e0 2024-08-19T13:19:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.960+0000 7fc8a77d8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a00b4430 con 0x7fc8a013c540 2024-08-19T13:19:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.960+0000 7fc8a77d8640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a006fd80 con 0x7fc8a006f6c0 2024-08-19T13:19:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.960+0000 7fc8a5574640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a006f2e0 0x7fc8a013db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.960+0000 7fc8a5574640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a006f2e0 0x7fc8a013db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52660/0 (socket says 172.21.15.103:52660) 2024-08-19T13:19:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.960+0000 7fc8a4d73640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8a006f6c0 0x7fc8a0135a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.960+0000 7fc8a5574640 1 -- 172.21.15.103:0/3395158298 learned_addr learned my addr 172.21.15.103:0/3395158298 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.961+0000 7fc8a5d75640 1 --2- 172.21.15.103:0/3395158298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8a013c540 0x7fc8a0135f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.961+0000 7fc8a4d73640 1 -- 172.21.15.103:0/3395158298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8a013c540 msgr2=0x7fc8a0135f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.961+0000 7fc8a4d73640 1 --2- 172.21.15.103:0/3395158298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8a013c540 0x7fc8a0135f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.961+0000 7fc8a4d73640 1 -- 172.21.15.103:0/3395158298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a006f2e0 msgr2=0x7fc8a013db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.961+0000 7fc8a4d73640 1 --2- 172.21.15.103:0/3395158298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a006f2e0 0x7fc8a013db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.961+0000 7fc8a4d73640 1 -- 172.21.15.103:0/3395158298 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a013f8b0 con 0x7fc8a006f6c0 2024-08-19T13:19:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.961+0000 7fc8a5574640 1 --2- 172.21.15.103:0/3395158298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a006f2e0 0x7fc8a013db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.961+0000 7fc8a4d73640 1 --2- 172.21.15.103:0/3395158298 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8a006f6c0 0x7fc8a0135a40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00c370 tx=0x7fc89c00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.961+0000 7fc895ffb640 1 -- 172.21.15.103:0/3395158298 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c0052d0 con 0x7fc8a006f6c0 2024-08-19T13:19:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.961+0000 7fc895ffb640 1 -- 172.21.15.103:0/3395158298 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc89c005490 con 0x7fc8a006f6c0 2024-08-19T13:19:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.961+0000 7fc895ffb640 1 -- 172.21.15.103:0/3395158298 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c007260 con 0x7fc8a006f6c0 2024-08-19T13:19:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.961+0000 7fc8a77d8640 1 -- 172.21.15.103:0/3395158298 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a013fb90 con 0x7fc8a006f6c0 2024-08-19T13:19:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.961+0000 7fc8a77d8640 1 -- 172.21.15.103:0/3395158298 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a01364c0 con 0x7fc8a006f6c0 2024-08-19T13:19:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.962+0000 7fc8a77d8640 1 -- 172.21.15.103:0/3395158298 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a0072240 con 0x7fc8a006f6c0 2024-08-19T13:19:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.963+0000 7fc895ffb640 1 -- 172.21.15.103:0/3395158298 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc89c007400 con 0x7fc8a006f6c0 2024-08-19T13:19:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.963+0000 7fc895ffb640 1 --2- 172.21.15.103:0/3395158298 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc88406be30 0x7fc88406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.963+0000 7fc895ffb640 1 -- 172.21.15.103:0/3395158298 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7fc89c08a1a0 con 0x7fc8a006f6c0 2024-08-19T13:19:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.963+0000 7fc8a5574640 1 --2- 172.21.15.103:0/3395158298 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc88406be30 0x7fc88406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.963+0000 7fc8a5574640 1 --2- 172.21.15.103:0/3395158298 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc88406be30 0x7fc88406e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc890006fd0 tx=0x7fc890008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:07.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:07.967+0000 7fc895ffb640 1 -- 172.21.15.103:0/3395158298 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fc89c0585d0 con 0x7fc8a006f6c0 2024-08-19T13:19:08.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.177+0000 7fc8a77d8640 1 -- 172.21.15.103:0/3395158298 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fc8a0137880 con 0x7fc88406be30 2024-08-19T13:19:08.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.178+0000 7fc895ffb640 1 -- 172.21.15.103:0/3395158298 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc8a0137880 con 0x7fc88406be30 2024-08-19T13:19:08.178 INFO:teuthology.orchestra.run.smithi103.stderr:instructed osd(s) 2 to deep-scrub 2024-08-19T13:19:08.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.180+0000 7fc8a77d8640 1 -- 172.21.15.103:0/3395158298 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc88406be30 msgr2=0x7fc88406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.180+0000 7fc8a77d8640 1 --2- 172.21.15.103:0/3395158298 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc88406be30 0x7fc88406e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc890006fd0 tx=0x7fc890008040 comp rx=0 tx=0).stop 2024-08-19T13:19:08.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.180+0000 7fc8a77d8640 1 -- 172.21.15.103:0/3395158298 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8a006f6c0 msgr2=0x7fc8a0135a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.180+0000 7fc8a77d8640 1 --2- 172.21.15.103:0/3395158298 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8a006f6c0 0x7fc8a0135a40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00c370 tx=0x7fc89c00c830 comp rx=0 tx=0).stop 2024-08-19T13:19:08.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.180+0000 7fc8a77d8640 1 -- 172.21.15.103:0/3395158298 shutdown_connections 2024-08-19T13:19:08.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.180+0000 7fc8a77d8640 1 --2- 172.21.15.103:0/3395158298 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc88406be30 0x7fc88406e2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.180+0000 7fc8a77d8640 1 --2- 172.21.15.103:0/3395158298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8a013c540 0x7fc8a0135f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.180+0000 7fc8a77d8640 1 --2- 172.21.15.103:0/3395158298 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8a006f6c0 0x7fc8a0135a40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.180+0000 7fc8a77d8640 1 --2- 172.21.15.103:0/3395158298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a006f2e0 0x7fc8a013db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.180+0000 7fc8a77d8640 1 -- 172.21.15.103:0/3395158298 >> 172.21.15.103:0/3395158298 conn(0x7fc8a00b3430 msgr2=0x7fc8a013b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:08.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.180+0000 7fc8a77d8640 1 -- 172.21.15.103:0/3395158298 shutdown_connections 2024-08-19T13:19:08.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.180+0000 7fc8a77d8640 1 -- 172.21.15.103:0/3395158298 wait complete. 2024-08-19T13:19:08.195 INFO:tasks.ceph:Scrubbing osd.3 2024-08-19T13:19:08.195 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-08-19T13:19:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.302+0000 7f953ae6e640 1 Processor -- start 2024-08-19T13:19:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.302+0000 7f953ae6e640 1 -- start start 2024-08-19T13:19:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.303+0000 7f953ae6e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9534071cc0 0x7f95341436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.303+0000 7f953ae6e640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f95340706b0 0x7f9534142b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.303+0000 7f953ae6e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95340adbb0 con 0x7f95340720a0 2024-08-19T13:19:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.303+0000 7f953ae6e640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f95340724f0 con 0x7f95340706b0 2024-08-19T13:19:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.303+0000 7f953ae6e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f95341431d0 con 0x7f9534071cc0 2024-08-19T13:19:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.303+0000 7f953940b640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f95340706b0 0x7f9534142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.303+0000 7f953940b640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f95340706b0 0x7f9534142b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52668/0 (socket says 172.21.15.103:52668) 2024-08-19T13:19:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.303+0000 7f953940b640 1 -- 172.21.15.103:0/3083998691 learned_addr learned my addr 172.21.15.103:0/3083998691 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.303+0000 7f9538c0a640 1 --2- 172.21.15.103:0/3083998691 >> v2:172.21.15.122:3300/0 conn(0x7f9534071cc0 0x7f95341436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.303+0000 7f953940b640 1 -- 172.21.15.103:0/3083998691 >> v2:172.21.15.122:3300/0 conn(0x7f9534071cc0 msgr2=0x7f95341436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.303+0000 7f953940b640 1 --2- 172.21.15.103:0/3083998691 >> v2:172.21.15.122:3300/0 conn(0x7f9534071cc0 0x7f95341436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.303+0000 7f953940b640 1 -- 172.21.15.103:0/3083998691 >> v1:172.21.15.122:6789/0 conn(0x7f95340720a0 legacy=0x7f953406ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.303+0000 7f953940b640 1 -- 172.21.15.103:0/3083998691 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953413fb00 con 0x7f95340706b0 2024-08-19T13:19:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.304+0000 7f953940b640 1 --2- 172.21.15.103:0/3083998691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95340706b0 0x7f9534142b00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9534073fe0 tx=0x7f951c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289ed588852f3fec server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.304+0000 7f95337fe640 1 -- 172.21.15.103:0/3083998691 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c037070 con 0x7f95340706b0 2024-08-19T13:19:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.304+0000 7f95337fe640 1 -- 172.21.15.103:0/3083998691 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951c032440 con 0x7f95340706b0 2024-08-19T13:19:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.304+0000 7f95337fe640 1 -- 172.21.15.103:0/3083998691 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c032810 con 0x7f95340706b0 2024-08-19T13:19:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.304+0000 7f953ae6e640 1 -- 172.21.15.103:0/3083998691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95340706b0 msgr2=0x7f9534142b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.304+0000 7f953ae6e640 1 --2- 172.21.15.103:0/3083998691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95340706b0 0x7f9534142b00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9534073fe0 tx=0x7f951c02a1a0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.305+0000 7f953ae6e640 1 -- 172.21.15.103:0/3083998691 shutdown_connections 2024-08-19T13:19:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.305+0000 7f953ae6e640 1 --2- 172.21.15.103:0/3083998691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95340706b0 0x7f9534142b00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.305+0000 7f953ae6e640 1 --2- 172.21.15.103:0/3083998691 >> v2:172.21.15.122:3300/0 conn(0x7f9534071cc0 0x7f95341436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.305+0000 7f953ae6e640 1 -- 172.21.15.103:0/3083998691 >> 172.21.15.103:0/3083998691 conn(0x7f95340b5370 msgr2=0x7f95340b5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.305+0000 7f953ae6e640 1 -- 172.21.15.103:0/3083998691 shutdown_connections 2024-08-19T13:19:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.305+0000 7f953ae6e640 1 -- 172.21.15.103:0/3083998691 wait complete. 2024-08-19T13:19:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.306+0000 7f953ae6e640 1 Processor -- start 2024-08-19T13:19:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.306+0000 7f953ae6e640 1 -- start start 2024-08-19T13:19:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.306+0000 7f953ae6e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9534071cc0 0x7f95341cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.306+0000 7f953ae6e640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f95340720a0 0x7f95341d85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.306+0000 7f953ae6e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95341cd380 0x7f95341cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.306+0000 7f953ae6e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9534140290 con 0x7f95341cd380 2024-08-19T13:19:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.306+0000 7f953ae6e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9534140110 con 0x7f9534071cc0 2024-08-19T13:19:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.306+0000 7f953ae6e640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9534140410 con 0x7f95340720a0 2024-08-19T13:19:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.306+0000 7f953940b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95341cd380 0x7f95341cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.306+0000 7f953940b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95341cd380 0x7f95341cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52676/0 (socket says 172.21.15.103:52676) 2024-08-19T13:19:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.306+0000 7f953940b640 1 -- 172.21.15.103:0/3855393601 learned_addr learned my addr 172.21.15.103:0/3855393601 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.306+0000 7f9533fff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f95340720a0 0x7f95341d85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.307+0000 7f9538c0a640 1 --2- 172.21.15.103:0/3855393601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9534071cc0 0x7f95341cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.307+0000 7f953940b640 1 -- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f95340720a0 msgr2=0x7f95341d85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.307+0000 7f953940b640 1 --2- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f95340720a0 0x7f95341d85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.307+0000 7f953940b640 1 -- 172.21.15.103:0/3855393601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9534071cc0 msgr2=0x7f95341cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.307+0000 7f953940b640 1 --2- 172.21.15.103:0/3855393601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9534071cc0 0x7f95341cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.307+0000 7f953940b640 1 -- 172.21.15.103:0/3855393601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95341cff40 con 0x7f95341cd380 2024-08-19T13:19:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.307+0000 7f953940b640 1 --2- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95341cd380 0x7f95341cd800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f951c009960 tx=0x7f951c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.307+0000 7f95317fa640 1 -- 172.21.15.103:0/3855393601 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c037040 con 0x7f95341cd380 2024-08-19T13:19:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.307+0000 7f95317fa640 1 -- 172.21.15.103:0/3855393601 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951c042070 con 0x7f95341cd380 2024-08-19T13:19:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.307+0000 7f95317fa640 1 -- 172.21.15.103:0/3855393601 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c030880 con 0x7f95341cd380 2024-08-19T13:19:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.307+0000 7f953ae6e640 1 -- 172.21.15.103:0/3855393601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95341cea00 con 0x7f95341cd380 2024-08-19T13:19:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.307+0000 7f953ae6e640 1 -- 172.21.15.103:0/3855393601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95341cc5b0 con 0x7f95341cd380 2024-08-19T13:19:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.308+0000 7f953ae6e640 1 -- 172.21.15.103:0/3855393601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9534142b00 con 0x7f95341cd380 2024-08-19T13:19:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.308+0000 7f95317fa640 1 -- 172.21.15.103:0/3855393601 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f951c030a20 con 0x7f95341cd380 2024-08-19T13:19:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.309+0000 7f95317fa640 1 --2- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f951406be30 0x7f951406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.309+0000 7f95317fa640 1 -- 172.21.15.103:0/3855393601 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7f951c0ab4f0 con 0x7f95341cd380 2024-08-19T13:19:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.309+0000 7f9538c0a640 1 --2- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f951406be30 0x7f951406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.309+0000 7f95317fa640 1 --2- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:6816/264232662,v1:172.21.15.103:6817/264232662] conn(0x7f9514072ff0 0x7f9514075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.309+0000 7f95317fa640 1 -- 172.21.15.103:0/3855393601 --> [v2:172.21.15.103:6816/264232662,v1:172.21.15.103:6817/264232662] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9514075ac0 con 0x7f9514072ff0 2024-08-19T13:19:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.309+0000 7f95317fa640 1 -- 172.21.15.103:0/3855393601 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f951c0b2050 con 0x7f95341cd380 2024-08-19T13:19:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.309+0000 7f9533fff640 1 --2- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:6816/264232662,v1:172.21.15.103:6817/264232662] conn(0x7f9514072ff0 0x7f9514075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.309+0000 7f9538c0a640 1 --2- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f951406be30 0x7f951406e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f952c009a60 tx=0x7f952c003990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.309+0000 7f9533fff640 1 --2- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:6816/264232662,v1:172.21.15.103:6817/264232662] conn(0x7f9514072ff0 0x7f9514075410 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.310+0000 7f95317fa640 1 -- 172.21.15.103:0/3855393601 <== osd.3 v2:172.21.15.103:6816/264232662 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9514075ac0 con 0x7f9514072ff0 2024-08-19T13:19:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.323+0000 7f953ae6e640 1 -- 172.21.15.103:0/3855393601 --> [v2:172.21.15.103:6816/264232662,v1:172.21.15.103:6817/264232662] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f95341cf680 con 0x7f9514072ff0 2024-08-19T13:19:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.323+0000 7f95317fa640 1 -- 172.21.15.103:0/3855393601 <== osd.3 v2:172.21.15.103:6816/264232662 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f95341cf680 con 0x7f9514072ff0 2024-08-19T13:19:08.324 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T13:19:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-19T13:19:08.324 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T13:19:08.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.323+0000 7f953ae6e640 1 -- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:6816/264232662,v1:172.21.15.103:6817/264232662] conn(0x7f9514072ff0 msgr2=0x7f9514075410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.323+0000 7f953ae6e640 1 --2- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:6816/264232662,v1:172.21.15.103:6817/264232662] conn(0x7f9514072ff0 0x7f9514075410 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.323+0000 7f953ae6e640 1 -- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f951406be30 msgr2=0x7f951406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.323+0000 7f953ae6e640 1 --2- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f951406be30 0x7f951406e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f952c009a60 tx=0x7f952c003990 comp rx=0 tx=0).stop 2024-08-19T13:19:08.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.323+0000 7f953ae6e640 1 -- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95341cd380 msgr2=0x7f95341cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.323+0000 7f953ae6e640 1 --2- 172.21.15.103:0/3855393601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95341cd380 0x7f95341cd800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f951c009960 tx=0x7f951c0046c0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.323+0000 7f953940b640 1 -- 172.21.15.103:0/3855393601 reap_dead start 2024-08-19T13:19:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.323+0000 7f953ae6e640 1 -- 172.21.15.103:0/3855393601 shutdown_connections 2024-08-19T13:19:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.323+0000 7f953ae6e640 1 -- 172.21.15.103:0/3855393601 >> 172.21.15.103:0/3855393601 conn(0x7f95340b5370 msgr2=0x7f95340b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.323+0000 7f953ae6e640 1 -- 172.21.15.103:0/3855393601 shutdown_connections 2024-08-19T13:19:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.323+0000 7f953ae6e640 1 -- 172.21.15.103:0/3855393601 wait complete. 2024-08-19T13:19:08.334 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-08-19T13:19:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.432+0000 7f422864e640 1 Processor -- start 2024-08-19T13:19:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.432+0000 7f422864e640 1 -- start start 2024-08-19T13:19:08.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.433+0000 7f422864e640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f422006ff20 0x7f4220142170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.433+0000 7f422864e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f422006f760 0x7f42201426b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.433+0000 7f422864e640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4220070300 con 0x7f4220146160 2024-08-19T13:19:08.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.433+0000 7f422864e640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f4220073a70 con 0x7f422006ff20 2024-08-19T13:19:08.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.433+0000 7f422864e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f4220073f00 con 0x7f422006f760 2024-08-19T13:19:08.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.433+0000 7f42263ea640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f422006ff20 0x7f4220142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.433+0000 7f42263ea640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f422006ff20 0x7f4220142170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52688/0 (socket says 172.21.15.103:52688) 2024-08-19T13:19:08.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.433+0000 7f42263ea640 1 -- 172.21.15.103:0/3262003112 learned_addr learned my addr 172.21.15.103:0/3262003112 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:08.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.434+0000 7f4225be9640 1 --2- 172.21.15.103:0/3262003112 >> v2:172.21.15.122:3300/0 conn(0x7f422006f760 0x7f42201426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.434+0000 7f42263ea640 1 -- 172.21.15.103:0/3262003112 >> v2:172.21.15.122:3300/0 conn(0x7f422006f760 msgr2=0x7f42201426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.434+0000 7f42263ea640 1 --2- 172.21.15.103:0/3262003112 >> v2:172.21.15.122:3300/0 conn(0x7f422006f760 0x7f42201426b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.434+0000 7f42263ea640 1 -- 172.21.15.103:0/3262003112 >> v1:172.21.15.103:6789/0 conn(0x7f4220146160 legacy=0x7f42201465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.434+0000 7f42263ea640 1 -- 172.21.15.103:0/3262003112 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4220072270 con 0x7f422006ff20 2024-08-19T13:19:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.434+0000 7f4225be9640 1 --2- 172.21.15.103:0/3262003112 >> v2:172.21.15.122:3300/0 conn(0x7f422006f760 0x7f42201426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T13:19:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.434+0000 7f42263ea640 1 --2- 172.21.15.103:0/3262003112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422006ff20 0x7f4220142170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4220071be0 tx=0x7f421c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1256cc1db8809c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.434+0000 7f42253e8640 1 -- 172.21.15.103:0/3262003112 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c037070 con 0x7f422006ff20 2024-08-19T13:19:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.435+0000 7f42253e8640 1 -- 172.21.15.103:0/3262003112 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f421c032440 con 0x7f422006ff20 2024-08-19T13:19:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.435+0000 7f42253e8640 1 -- 172.21.15.103:0/3262003112 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c032810 con 0x7f422006ff20 2024-08-19T13:19:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.435+0000 7f422864e640 1 -- 172.21.15.103:0/3262003112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422006ff20 msgr2=0x7f4220142170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.435+0000 7f422864e640 1 --2- 172.21.15.103:0/3262003112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422006ff20 0x7f4220142170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4220071be0 tx=0x7f421c02a1e0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.435+0000 7f422864e640 1 -- 172.21.15.103:0/3262003112 shutdown_connections 2024-08-19T13:19:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.435+0000 7f422864e640 1 --2- 172.21.15.103:0/3262003112 >> v2:172.21.15.122:3300/0 conn(0x7f422006f760 0x7f42201426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.435+0000 7f422864e640 1 --2- 172.21.15.103:0/3262003112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422006ff20 0x7f4220142170 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.435+0000 7f422864e640 1 -- 172.21.15.103:0/3262003112 >> 172.21.15.103:0/3262003112 conn(0x7f42200b6bb0 msgr2=0x7f42200b6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:08.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.435+0000 7f422864e640 1 -- 172.21.15.103:0/3262003112 shutdown_connections 2024-08-19T13:19:08.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.435+0000 7f422864e640 1 -- 172.21.15.103:0/3262003112 wait complete. 2024-08-19T13:19:08.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.436+0000 7f422864e640 1 Processor -- start 2024-08-19T13:19:08.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.436+0000 7f422864e640 1 -- start start 2024-08-19T13:19:08.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f422864e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422006f760 0x7f4220145890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f422864e640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f422006ff20 0x7f4220137c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f422864e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4220146160 0x7f4220138180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f422864e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f422006f500 con 0x7f422006f760 2024-08-19T13:19:08.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f422864e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f422006f380 con 0x7f4220146160 2024-08-19T13:19:08.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f422864e640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f422014d4b0 con 0x7f422006ff20 2024-08-19T13:19:08.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f42263ea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422006f760 0x7f4220145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f4225be9640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f422006ff20 0x7f4220137c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f42263ea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422006f760 0x7f4220145890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52698/0 (socket says 172.21.15.103:52698) 2024-08-19T13:19:08.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f42263ea640 1 -- 172.21.15.103:0/760801710 learned_addr learned my addr 172.21.15.103:0/760801710 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:08.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f4225be9640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f422006ff20 0x7f4220137c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41606/0 (socket says 172.21.15.103:41606) 2024-08-19T13:19:08.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f4226beb640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4220146160 0x7f4220138180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f4225be9640 1 -- 172.21.15.103:0/760801710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4220146160 msgr2=0x7f4220138180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f4225be9640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4220146160 0x7f4220138180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f4225be9640 1 -- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422006f760 msgr2=0x7f4220145890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f4225be9640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422006f760 0x7f4220145890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f4225be9640 1 -- 172.21.15.103:0/760801710 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42201377b0 con 0x7f422006ff20 2024-08-19T13:19:08.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f42263ea640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422006f760 0x7f4220145890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:08.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f4225be9640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f422006ff20 0x7f4220137c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f421000c820 tx=0x7f421000cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:08.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f4216ffd640 1 -- 172.21.15.103:0/760801710 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210015070 con 0x7f422006ff20 2024-08-19T13:19:08.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.437+0000 7f4216ffd640 1 -- 172.21.15.103:0/760801710 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4210004220 con 0x7f422006ff20 2024-08-19T13:19:08.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.438+0000 7f422864e640 1 -- 172.21.15.103:0/760801710 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4220144f80 con 0x7f422006ff20 2024-08-19T13:19:08.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.438+0000 7f4216ffd640 1 -- 172.21.15.103:0/760801710 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421000ce10 con 0x7f422006ff20 2024-08-19T13:19:08.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.438+0000 7f422864e640 1 -- 172.21.15.103:0/760801710 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f42201386c0 con 0x7f422006ff20 2024-08-19T13:19:08.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.439+0000 7f422864e640 1 -- 172.21.15.103:0/760801710 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f422006e930 con 0x7f422006ff20 2024-08-19T13:19:08.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.439+0000 7f4216ffd640 1 -- 172.21.15.103:0/760801710 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4210007830 con 0x7f422006ff20 2024-08-19T13:19:08.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.439+0000 7f4216ffd640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f420406be60 0x7f420406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.439+0000 7f4216ffd640 1 -- 172.21.15.103:0/760801710 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7f421008ab40 con 0x7f422006ff20 2024-08-19T13:19:08.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.439+0000 7f42263ea640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f420406be60 0x7f420406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.440+0000 7f42263ea640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f420406be60 0x7f420406e320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f421c035040 tx=0x7f421c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:08.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.443+0000 7f4216ffd640 1 -- 172.21.15.103:0/760801710 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f4210058f70 con 0x7f422006ff20 2024-08-19T13:19:08.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.608+0000 7f422864e640 1 -- 172.21.15.103:0/760801710 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f42201422c0 con 0x7f420406be60 2024-08-19T13:19:08.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.608+0000 7f4216ffd640 1 -- 172.21.15.103:0/760801710 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f42201422c0 con 0x7f420406be60 2024-08-19T13:19:08.609 INFO:teuthology.orchestra.run.smithi103.stderr:instructed osd(s) 3 to deep-scrub 2024-08-19T13:19:08.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.610+0000 7f422864e640 1 -- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f420406be60 msgr2=0x7f420406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.610+0000 7f422864e640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f420406be60 0x7f420406e320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f421c035040 tx=0x7f421c02e000 comp rx=0 tx=0).stop 2024-08-19T13:19:08.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.610+0000 7f422864e640 1 -- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f422006ff20 msgr2=0x7f4220137c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.610+0000 7f422864e640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f422006ff20 0x7f4220137c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f421000c820 tx=0x7f421000cce0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.610+0000 7f422864e640 1 -- 172.21.15.103:0/760801710 shutdown_connections 2024-08-19T13:19:08.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.610+0000 7f422864e640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f420406be60 0x7f420406e320 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.610+0000 7f422864e640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4220146160 0x7f4220138180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.610+0000 7f422864e640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f422006ff20 0x7f4220137c40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.610+0000 7f422864e640 1 --2- 172.21.15.103:0/760801710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422006f760 0x7f4220145890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.610+0000 7f422864e640 1 -- 172.21.15.103:0/760801710 >> 172.21.15.103:0/760801710 conn(0x7f42200b6bb0 msgr2=0x7f42200b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:08.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.610+0000 7f422864e640 1 -- 172.21.15.103:0/760801710 shutdown_connections 2024-08-19T13:19:08.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.610+0000 7f422864e640 1 -- 172.21.15.103:0/760801710 wait complete. 2024-08-19T13:19:08.621 INFO:tasks.ceph:Scrubbing osd.4 2024-08-19T13:19:08.622 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-08-19T13:19:08.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.724+0000 7fdb37d54640 1 Processor -- start 2024-08-19T13:19:08.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.724+0000 7fdb37d54640 1 -- start start 2024-08-19T13:19:08.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.725+0000 7fdb37d54640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdb30071cc0 0x7fdb301436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.725+0000 7fdb37d54640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fdb300720a0 0x7fdb30142b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.725+0000 7fdb37d54640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb300724c0 con 0x7fdb3006ffa0 2024-08-19T13:19:08.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.725+0000 7fdb37d54640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fdb300adbb0 con 0x7fdb300720a0 2024-08-19T13:19:08.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.725+0000 7fdb37d54640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fdb301431d0 con 0x7fdb30071cc0 2024-08-19T13:19:08.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.725+0000 7fdb35af0640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdb30071cc0 0x7fdb301436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.725+0000 7fdb35af0640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdb30071cc0 0x7fdb301436f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41622/0 (socket says 172.21.15.103:41622) 2024-08-19T13:19:08.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.725+0000 7fdb35af0640 1 -- 172.21.15.103:0/4257337267 learned_addr learned my addr 172.21.15.103:0/4257337267 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:08.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.725+0000 7fdb35af0640 1 -- 172.21.15.103:0/4257337267 >> v2:172.21.15.122:3300/0 conn(0x7fdb300720a0 msgr2=0x7fdb30142b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:08.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.725+0000 7fdb352ef640 1 --2- 172.21.15.103:0/4257337267 >> v2:172.21.15.122:3300/0 conn(0x7fdb300720a0 0x7fdb30142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.725+0000 7fdb35af0640 1 --2- 172.21.15.103:0/4257337267 >> v2:172.21.15.122:3300/0 conn(0x7fdb300720a0 0x7fdb30142b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.725+0000 7fdb35af0640 1 -- 172.21.15.103:0/4257337267 >> v1:172.21.15.122:6789/0 conn(0x7fdb3006ffa0 legacy=0x7fdb3013fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:08.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.725+0000 7fdb35af0640 1 -- 172.21.15.103:0/4257337267 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb30140210 con 0x7fdb30071cc0 2024-08-19T13:19:08.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.726+0000 7fdb35af0640 1 --2- 172.21.15.103:0/4257337267 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdb30071cc0 0x7fdb301436f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009870 tx=0x7fdb2c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce14e5df43aef356 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:08.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.726+0000 7fdb34aee640 1 -- 172.21.15.103:0/4257337267 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c037070 con 0x7fdb30071cc0 2024-08-19T13:19:08.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.726+0000 7fdb34aee640 1 -- 172.21.15.103:0/4257337267 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb2c032440 con 0x7fdb30071cc0 2024-08-19T13:19:08.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.726+0000 7fdb34aee640 1 -- 172.21.15.103:0/4257337267 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c032810 con 0x7fdb30071cc0 2024-08-19T13:19:08.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.726+0000 7fdb37d54640 1 -- 172.21.15.103:0/4257337267 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdb30071cc0 msgr2=0x7fdb301436f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.726+0000 7fdb37d54640 1 --2- 172.21.15.103:0/4257337267 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdb30071cc0 0x7fdb301436f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009870 tx=0x7fdb2c02a210 comp rx=0 tx=0).stop 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.726+0000 7fdb37d54640 1 -- 172.21.15.103:0/4257337267 shutdown_connections 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.726+0000 7fdb37d54640 1 --2- 172.21.15.103:0/4257337267 >> v2:172.21.15.122:3300/0 conn(0x7fdb300720a0 0x7fdb30142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.726+0000 7fdb37d54640 1 --2- 172.21.15.103:0/4257337267 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdb30071cc0 0x7fdb301436f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.726+0000 7fdb37d54640 1 -- 172.21.15.103:0/4257337267 >> 172.21.15.103:0/4257337267 conn(0x7fdb300b5370 msgr2=0x7fdb300b5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.727+0000 7fdb37d54640 1 -- 172.21.15.103:0/4257337267 shutdown_connections 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.727+0000 7fdb37d54640 1 -- 172.21.15.103:0/4257337267 wait complete. 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.727+0000 7fdb37d54640 1 Processor -- start 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb37d54640 1 -- start start 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb37d54640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb3006ffa0 0x7fdb30146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb37d54640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdb30071cc0 0x7fdb301db890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb37d54640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb300720a0 0x7fdb30134950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb37d54640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30070800 con 0x7fdb300720a0 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb37d54640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30070680 con 0x7fdb3006ffa0 2024-08-19T13:19:08.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb37d54640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fdb3014d410 con 0x7fdb30071cc0 2024-08-19T13:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb352ef640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdb30071cc0 0x7fdb301db890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb352ef640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdb30071cc0 0x7fdb301db890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41638/0 (socket says 172.21.15.103:41638) 2024-08-19T13:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb352ef640 1 -- 172.21.15.103:0/4181556254 learned_addr learned my addr 172.21.15.103:0/4181556254 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb362f1640 1 --2- 172.21.15.103:0/4181556254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb300720a0 0x7fdb30134950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb35af0640 1 --2- 172.21.15.103:0/4181556254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb3006ffa0 0x7fdb30146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb352ef640 1 -- 172.21.15.103:0/4181556254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb3006ffa0 msgr2=0x7fdb30146340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb352ef640 1 --2- 172.21.15.103:0/4181556254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb3006ffa0 0x7fdb30146340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb352ef640 1 -- 172.21.15.103:0/4181556254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb300720a0 msgr2=0x7fdb30134950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb352ef640 1 --2- 172.21.15.103:0/4181556254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb300720a0 0x7fdb30134950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.728+0000 7fdb352ef640 1 -- 172.21.15.103:0/4181556254 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb301dbdd0 con 0x7fdb30071cc0 2024-08-19T13:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.729+0000 7fdb352ef640 1 --2- 172.21.15.103:0/4181556254 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdb30071cc0 0x7fdb301db890 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000c7f0 tx=0x7fdb2000ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.729+0000 7fdb267fc640 1 -- 172.21.15.103:0/4181556254 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20004230 con 0x7fdb30071cc0 2024-08-19T13:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.729+0000 7fdb267fc640 1 -- 172.21.15.103:0/4181556254 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb200043d0 con 0x7fdb30071cc0 2024-08-19T13:19:08.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.729+0000 7fdb267fc640 1 -- 172.21.15.103:0/4181556254 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20007860 con 0x7fdb30071cc0 2024-08-19T13:19:08.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.729+0000 7fdb37d54640 1 -- 172.21.15.103:0/4181556254 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb30142c20 con 0x7fdb30071cc0 2024-08-19T13:19:08.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.729+0000 7fdb37d54640 1 -- 172.21.15.103:0/4181556254 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb301dc2d0 con 0x7fdb30071cc0 2024-08-19T13:19:08.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.730+0000 7fdb37d54640 1 -- 172.21.15.103:0/4181556254 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb301436f0 con 0x7fdb30071cc0 2024-08-19T13:19:08.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.730+0000 7fdb267fc640 1 -- 172.21.15.103:0/4181556254 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb20007c10 con 0x7fdb30071cc0 2024-08-19T13:19:08.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.731+0000 7fdb267fc640 1 --2- 172.21.15.103:0/4181556254 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fdb1406be30 0x7fdb1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.731+0000 7fdb267fc640 1 -- 172.21.15.103:0/4181556254 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7fdb2008ae60 con 0x7fdb30071cc0 2024-08-19T13:19:08.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.731+0000 7fdb35af0640 1 --2- 172.21.15.103:0/4181556254 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fdb1406be30 0x7fdb1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.731+0000 7fdb267fc640 1 --2- 172.21.15.103:0/4181556254 >> [v2:172.21.15.122:6800/634629443,v1:172.21.15.122:6801/634629443] conn(0x7fdb14072ff0 0x7fdb14075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.731+0000 7fdb267fc640 1 -- 172.21.15.103:0/4181556254 --> [v2:172.21.15.122:6800/634629443,v1:172.21.15.122:6801/634629443] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb20018ec0 con 0x7fdb14072ff0 2024-08-19T13:19:08.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.731+0000 7fdb267fc640 1 -- 172.21.15.103:0/4181556254 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdb2008d560 con 0x7fdb30071cc0 2024-08-19T13:19:08.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.731+0000 7fdb362f1640 1 --2- 172.21.15.103:0/4181556254 >> [v2:172.21.15.122:6800/634629443,v1:172.21.15.122:6801/634629443] conn(0x7fdb14072ff0 0x7fdb14075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.731+0000 7fdb35af0640 1 --2- 172.21.15.103:0/4181556254 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fdb1406be30 0x7fdb1406e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c034000 tx=0x7fdb2c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:08.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.731+0000 7fdb362f1640 1 --2- 172.21.15.103:0/4181556254 >> [v2:172.21.15.122:6800/634629443,v1:172.21.15.122:6801/634629443] conn(0x7fdb14072ff0 0x7fdb14075410 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:08.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.732+0000 7fdb267fc640 1 -- 172.21.15.103:0/4181556254 <== osd.4 v2:172.21.15.122:6800/634629443 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb20018ec0 con 0x7fdb14072ff0 2024-08-19T13:19:08.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.744+0000 7fdb37d54640 1 -- 172.21.15.103:0/4181556254 --> [v2:172.21.15.122:6800/634629443,v1:172.21.15.122:6801/634629443] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdb301350e0 con 0x7fdb14072ff0 2024-08-19T13:19:08.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.745+0000 7fdb267fc640 1 -- 172.21.15.103:0/4181556254 <== osd.4 v2:172.21.15.122:6800/634629443 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdb301350e0 con 0x7fdb14072ff0 2024-08-19T13:19:08.746 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T13:19:08.746 INFO:teuthology.orchestra.run.smithi103.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-19T13:19:08.746 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T13:19:08.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.745+0000 7fdb37d54640 1 -- 172.21.15.103:0/4181556254 >> [v2:172.21.15.122:6800/634629443,v1:172.21.15.122:6801/634629443] conn(0x7fdb14072ff0 msgr2=0x7fdb14075410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.745+0000 7fdb37d54640 1 --2- 172.21.15.103:0/4181556254 >> [v2:172.21.15.122:6800/634629443,v1:172.21.15.122:6801/634629443] conn(0x7fdb14072ff0 0x7fdb14075410 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.746+0000 7fdb37d54640 1 -- 172.21.15.103:0/4181556254 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fdb1406be30 msgr2=0x7fdb1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.746+0000 7fdb37d54640 1 --2- 172.21.15.103:0/4181556254 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fdb1406be30 0x7fdb1406e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c034000 tx=0x7fdb2c02e000 comp rx=0 tx=0).stop 2024-08-19T13:19:08.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.746+0000 7fdb37d54640 1 -- 172.21.15.103:0/4181556254 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdb30071cc0 msgr2=0x7fdb301db890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.746+0000 7fdb37d54640 1 --2- 172.21.15.103:0/4181556254 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdb30071cc0 0x7fdb301db890 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000c7f0 tx=0x7fdb2000ccb0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.746+0000 7fdb362f1640 1 -- 172.21.15.103:0/4181556254 reap_dead start 2024-08-19T13:19:08.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.746+0000 7fdb37d54640 1 -- 172.21.15.103:0/4181556254 shutdown_connections 2024-08-19T13:19:08.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.746+0000 7fdb37d54640 1 -- 172.21.15.103:0/4181556254 >> 172.21.15.103:0/4181556254 conn(0x7fdb300b5370 msgr2=0x7fdb30072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:08.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.746+0000 7fdb37d54640 1 -- 172.21.15.103:0/4181556254 shutdown_connections 2024-08-19T13:19:08.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.746+0000 7fdb37d54640 1 -- 172.21.15.103:0/4181556254 wait complete. 2024-08-19T13:19:08.757 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-08-19T13:19:08.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.855+0000 7fc9c5133640 1 Processor -- start 2024-08-19T13:19:08.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.855+0000 7fc9c5133640 1 -- start start 2024-08-19T13:19:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.856+0000 7fc9c5133640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc9c0073a10 0x7fc9c0142b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.856+0000 7fc9c5133640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9c01430c0 0x7fc9c014d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.856+0000 7fc9c5133640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c0142880 con 0x7fc9c0072a30 2024-08-19T13:19:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.856+0000 7fc9c5133640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc9c006f310 con 0x7fc9c0073a10 2024-08-19T13:19:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.856+0000 7fc9c5133640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c00b0d20 con 0x7fc9c01430c0 2024-08-19T13:19:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.856+0000 7fc9bf59e640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9c01430c0 0x7fc9c014d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.856+0000 7fc9bed9d640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fc9c0072a30 0x7fc9c0142170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:42962/0 (socket says 172.21.15.103:42962) 2024-08-19T13:19:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.856+0000 7fc9bed9d640 1 -- 172.21.15.103:0/2312967721 learned_addr learned my addr 172.21.15.103:0/2312967721 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:19:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.856+0000 7fc9be59c640 1 --2- 172.21.15.103:0/2312967721 >> v2:172.21.15.122:3300/0 conn(0x7fc9c0073a10 0x7fc9c0142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.856+0000 7fc9bf59e640 1 -- 172.21.15.103:0/2312967721 >> v2:172.21.15.122:3300/0 conn(0x7fc9c0073a10 msgr2=0x7fc9c0142b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:08.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.856+0000 7fc9bf59e640 1 --2- 172.21.15.103:0/2312967721 >> v2:172.21.15.122:3300/0 conn(0x7fc9c0073a10 0x7fc9c0142b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.856+0000 7fc9bdd9b640 1 -- 172.21.15.103:0/2312967721 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1609987214 0 0) 0x7fc9c0142880 con 0x7fc9c0072a30 2024-08-19T13:19:08.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.856+0000 7fc9bf59e640 1 -- 172.21.15.103:0/2312967721 >> v1:172.21.15.103:6789/0 conn(0x7fc9c0072a30 legacy=0x7fc9c0142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.856+0000 7fc9bf59e640 1 -- 172.21.15.103:0/2312967721 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c0073470 con 0x7fc9c01430c0 2024-08-19T13:19:08.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.857+0000 7fc9bf59e640 1 --2- 172.21.15.103:0/2312967721 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9c01430c0 0x7fc9c014d450 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009d90 tx=0x7fc9b802a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a290c4bb129e576 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:08.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.857+0000 7fc9bdd9b640 1 -- 172.21.15.103:0/2312967721 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8007740 con 0x7fc9c01430c0 2024-08-19T13:19:08.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.857+0000 7fc9bdd9b640 1 -- 172.21.15.103:0/2312967721 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b80078e0 con 0x7fc9c01430c0 2024-08-19T13:19:08.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.857+0000 7fc9bdd9b640 1 -- 172.21.15.103:0/2312967721 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8007cb0 con 0x7fc9c01430c0 2024-08-19T13:19:08.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.858+0000 7fc9c5133640 1 -- 172.21.15.103:0/2312967721 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9c01430c0 msgr2=0x7fc9c014d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.858+0000 7fc9c5133640 1 --2- 172.21.15.103:0/2312967721 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9c01430c0 0x7fc9c014d450 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009d90 tx=0x7fc9b802a530 comp rx=0 tx=0).stop 2024-08-19T13:19:08.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.858+0000 7fc9c5133640 1 -- 172.21.15.103:0/2312967721 shutdown_connections 2024-08-19T13:19:08.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.858+0000 7fc9c5133640 1 --2- 172.21.15.103:0/2312967721 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9c01430c0 0x7fc9c014d450 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.858+0000 7fc9c5133640 1 --2- 172.21.15.103:0/2312967721 >> v2:172.21.15.122:3300/0 conn(0x7fc9c0073a10 0x7fc9c0142b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.858+0000 7fc9c5133640 1 -- 172.21.15.103:0/2312967721 >> 172.21.15.103:0/2312967721 conn(0x7fc9c00b6b10 msgr2=0x7fc9c00b6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:08.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.858+0000 7fc9c5133640 1 -- 172.21.15.103:0/2312967721 shutdown_connections 2024-08-19T13:19:08.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.858+0000 7fc9c5133640 1 -- 172.21.15.103:0/2312967721 wait complete. 2024-08-19T13:19:08.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.859+0000 7fc9c5133640 1 Processor -- start 2024-08-19T13:19:08.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.859+0000 7fc9c5133640 1 -- start start 2024-08-19T13:19:08.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9c5133640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9c0072a30 0x7fc9c00b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9c5133640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9c0073a10 0x7fc9c01db5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9c5133640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9c01430c0 0x7fc9c0135af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9c5133640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01436b0 con 0x7fc9c01430c0 2024-08-19T13:19:08.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9c5133640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c0143530 con 0x7fc9c0073a10 2024-08-19T13:19:08.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9c5133640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c014d990 con 0x7fc9c0072a30 2024-08-19T13:19:08.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9bf59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9c01430c0 0x7fc9c0135af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9bed9d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9c0072a30 0x7fc9c00b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9bf59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9c01430c0 0x7fc9c0135af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52710/0 (socket says 172.21.15.103:52710) 2024-08-19T13:19:08.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9bed9d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9c0072a30 0x7fc9c00b0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41642/0 (socket says 172.21.15.103:41642) 2024-08-19T13:19:08.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9bf59e640 1 -- 172.21.15.103:0/3739323464 learned_addr learned my addr 172.21.15.103:0/3739323464 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:08.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9be59c640 1 --2- 172.21.15.103:0/3739323464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9c0073a10 0x7fc9c01db5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9bf59e640 1 -- 172.21.15.103:0/3739323464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9c0072a30 msgr2=0x7fc9c00b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9bf59e640 1 --2- 172.21.15.103:0/3739323464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9c0072a30 0x7fc9c00b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9bf59e640 1 -- 172.21.15.103:0/3739323464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9c0073a10 msgr2=0x7fc9c01db5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:08.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9bf59e640 1 --2- 172.21.15.103:0/3739323464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9c0073a10 0x7fc9c01db5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:08.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9bf59e640 1 -- 172.21.15.103:0/3739323464 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c0138da0 con 0x7fc9c01430c0 2024-08-19T13:19:08.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9bed9d640 1 --2- 172.21.15.103:0/3739323464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9c0072a30 0x7fc9c00b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:08.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.860+0000 7fc9bf59e640 1 --2- 172.21.15.103:0/3739323464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9c01430c0 0x7fc9c0135af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009eb0 tx=0x7fc9b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:08.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.861+0000 7fc9b77fe640 1 -- 172.21.15.103:0/3739323464 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b803f070 con 0x7fc9c01430c0 2024-08-19T13:19:08.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.861+0000 7fc9b77fe640 1 -- 172.21.15.103:0/3739323464 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b8031bc0 con 0x7fc9c01430c0 2024-08-19T13:19:08.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.861+0000 7fc9c5133640 1 -- 172.21.15.103:0/3739323464 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c01dbd60 con 0x7fc9c01430c0 2024-08-19T13:19:08.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.861+0000 7fc9b77fe640 1 -- 172.21.15.103:0/3739323464 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8044460 con 0x7fc9c01430c0 2024-08-19T13:19:08.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.861+0000 7fc9c5133640 1 -- 172.21.15.103:0/3739323464 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c0136030 con 0x7fc9c01430c0 2024-08-19T13:19:08.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.862+0000 7fc9c5133640 1 -- 172.21.15.103:0/3739323464 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c0142b80 con 0x7fc9c01430c0 2024-08-19T13:19:08.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.862+0000 7fc9b77fe640 1 -- 172.21.15.103:0/3739323464 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc9b8007790 con 0x7fc9c01430c0 2024-08-19T13:19:08.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.863+0000 7fc9b77fe640 1 --2- 172.21.15.103:0/3739323464 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc9a006be60 0x7fc9a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:08.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.863+0000 7fc9b77fe640 1 -- 172.21.15.103:0/3739323464 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7fc9b80ac4f0 con 0x7fc9c01430c0 2024-08-19T13:19:08.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.863+0000 7fc9bed9d640 1 --2- 172.21.15.103:0/3739323464 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc9a006be60 0x7fc9a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:08.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.863+0000 7fc9bed9d640 1 --2- 172.21.15.103:0/3739323464 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc9a006be60 0x7fc9a006e320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0000f60 tx=0x7fc9b0007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:08.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:08.867+0000 7fc9b77fe640 1 -- 172.21.15.103:0/3739323464 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fc9b807a920 con 0x7fc9c01430c0 2024-08-19T13:19:09.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.057+0000 7fc9c5133640 1 -- 172.21.15.103:0/3739323464 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fc9c0075a00 con 0x7fc9a006be60 2024-08-19T13:19:09.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.057+0000 7fc9b77fe640 1 -- 172.21.15.103:0/3739323464 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc9b0001cd0 con 0x7fc9a006be60 2024-08-19T13:19:09.058 INFO:teuthology.orchestra.run.smithi103.stderr:instructed osd(s) 4 to deep-scrub 2024-08-19T13:19:09.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.059+0000 7fc9c5133640 1 -- 172.21.15.103:0/3739323464 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc9a006be60 msgr2=0x7fc9a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.059+0000 7fc9c5133640 1 --2- 172.21.15.103:0/3739323464 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc9a006be60 0x7fc9a006e320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0000f60 tx=0x7fc9b0007d50 comp rx=0 tx=0).stop 2024-08-19T13:19:09.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.059+0000 7fc9c5133640 1 -- 172.21.15.103:0/3739323464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9c01430c0 msgr2=0x7fc9c0135af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.059+0000 7fc9c5133640 1 --2- 172.21.15.103:0/3739323464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9c01430c0 0x7fc9c0135af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009eb0 tx=0x7fc9b8004a40 comp rx=0 tx=0).stop 2024-08-19T13:19:09.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.059+0000 7fc9c5133640 1 -- 172.21.15.103:0/3739323464 shutdown_connections 2024-08-19T13:19:09.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.059+0000 7fc9c5133640 1 --2- 172.21.15.103:0/3739323464 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fc9a006be60 0x7fc9a006e320 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.059+0000 7fc9c5133640 1 --2- 172.21.15.103:0/3739323464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9c01430c0 0x7fc9c0135af0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.059+0000 7fc9c5133640 1 --2- 172.21.15.103:0/3739323464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9c0073a10 0x7fc9c01db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.059+0000 7fc9c5133640 1 --2- 172.21.15.103:0/3739323464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9c0072a30 0x7fc9c00b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.059+0000 7fc9c5133640 1 -- 172.21.15.103:0/3739323464 >> 172.21.15.103:0/3739323464 conn(0x7fc9c00b6b10 msgr2=0x7fc9c01413e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:09.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.059+0000 7fc9c5133640 1 -- 172.21.15.103:0/3739323464 shutdown_connections 2024-08-19T13:19:09.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.059+0000 7fc9c5133640 1 -- 172.21.15.103:0/3739323464 wait complete. 2024-08-19T13:19:09.072 INFO:tasks.ceph:Scrubbing osd.5 2024-08-19T13:19:09.072 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-08-19T13:19:09.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.183+0000 7f0d570fc640 1 Processor -- start 2024-08-19T13:19:09.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.183+0000 7f0d570fc640 1 -- start start 2024-08-19T13:19:09.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.183+0000 7f0d570fc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0d5013f2a0 0x7f0d50143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.183+0000 7f0d570fc640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d500aa410 con 0x7f0d50070a50 2024-08-19T13:19:09.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.183+0000 7f0d570fc640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d50073e30 con 0x7f0d50142f10 2024-08-19T13:19:09.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.183+0000 7f0d570fc640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f0d50140730 con 0x7f0d5013f2a0 2024-08-19T13:19:09.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.184+0000 7f0d4ffff640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f0d50070a50 0x7f0d5013eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:42968/0 (socket says 172.21.15.103:42968) 2024-08-19T13:19:09.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.184+0000 7f0d4ffff640 1 -- 172.21.15.103:0/2915499869 learned_addr learned my addr 172.21.15.103:0/2915499869 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:19:09.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.184+0000 7f0d55699640 1 --2- 172.21.15.103:0/2915499869 >> v2:172.21.15.122:3300/0 conn(0x7f0d5013f2a0 0x7f0d50143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.184+0000 7f0d4f7fe640 1 -- 172.21.15.103:0/2915499869 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1235022906 0 0) 0x7f0d500aa410 con 0x7f0d50070a50 2024-08-19T13:19:09.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.184+0000 7f0d4f7fe640 1 -- 172.21.15.103:0/2915499869 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d40003520 con 0x7f0d50070a50 2024-08-19T13:19:09.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.184+0000 7f0d55699640 1 -- 172.21.15.103:0/2915499869 >> v1:172.21.15.122:6789/0 conn(0x7f0d50142f10 legacy=0x7f0d50070340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.184+0000 7f0d55699640 1 -- 172.21.15.103:0/2915499869 >> v1:172.21.15.103:6789/0 conn(0x7f0d50070a50 legacy=0x7f0d5013eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.184+0000 7f0d55699640 1 -- 172.21.15.103:0/2915499869 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d501432f0 con 0x7f0d5013f2a0 2024-08-19T13:19:09.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.185+0000 7f0d55699640 1 --2- 172.21.15.103:0/2915499869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d5013f2a0 0x7f0d50143860 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009d90 tx=0x7f0d4802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=203c7059bef6eabc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:09.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.185+0000 7f0d4f7fe640 1 -- 172.21.15.103:0/2915499869 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48007740 con 0x7f0d5013f2a0 2024-08-19T13:19:09.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.185+0000 7f0d4f7fe640 1 -- 172.21.15.103:0/2915499869 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d480078e0 con 0x7f0d5013f2a0 2024-08-19T13:19:09.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.185+0000 7f0d4f7fe640 1 -- 172.21.15.103:0/2915499869 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48007cb0 con 0x7f0d5013f2a0 2024-08-19T13:19:09.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.186+0000 7f0d570fc640 1 -- 172.21.15.103:0/2915499869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d5013f2a0 msgr2=0x7f0d50143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.186+0000 7f0d570fc640 1 --2- 172.21.15.103:0/2915499869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d5013f2a0 0x7f0d50143860 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009d90 tx=0x7f0d4802a660 comp rx=0 tx=0).stop 2024-08-19T13:19:09.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.186+0000 7f0d570fc640 1 -- 172.21.15.103:0/2915499869 shutdown_connections 2024-08-19T13:19:09.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.186+0000 7f0d570fc640 1 --2- 172.21.15.103:0/2915499869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d5013f2a0 0x7f0d50143860 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.186+0000 7f0d570fc640 1 -- 172.21.15.103:0/2915499869 >> 172.21.15.103:0/2915499869 conn(0x7f0d50067b10 msgr2=0x7f0d50067f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:09.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.186+0000 7f0d570fc640 1 -- 172.21.15.103:0/2915499869 shutdown_connections 2024-08-19T13:19:09.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.186+0000 7f0d570fc640 1 -- 172.21.15.103:0/2915499869 wait complete. 2024-08-19T13:19:09.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.187+0000 7f0d570fc640 1 Processor -- start 2024-08-19T13:19:09.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.187+0000 7f0d570fc640 1 -- start start 2024-08-19T13:19:09.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d570fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d5013f2a0 0x7f0d50141990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d570fc640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0d50142f10 0x7f0d501420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d570fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d50148410 0x7f0d50142620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d570fc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d500a94c0 con 0x7f0d5013f2a0 2024-08-19T13:19:09.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d570fc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5006c570 con 0x7f0d50148410 2024-08-19T13:19:09.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d570fc640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5014ce20 con 0x7f0d50142f10 2024-08-19T13:19:09.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d4ffff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0d50142f10 0x7f0d501420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d54e98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d5013f2a0 0x7f0d50141990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d55699640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d50148410 0x7f0d50142620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d4ffff640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0d50142f10 0x7f0d501420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41648/0 (socket says 172.21.15.103:41648) 2024-08-19T13:19:09.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d54e98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d5013f2a0 0x7f0d50141990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52714/0 (socket says 172.21.15.103:52714) 2024-08-19T13:19:09.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d4ffff640 1 -- 172.21.15.103:0/2758162619 learned_addr learned my addr 172.21.15.103:0/2758162619 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:09.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d55699640 1 -- 172.21.15.103:0/2758162619 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0d50142f10 msgr2=0x7f0d501420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d55699640 1 --2- 172.21.15.103:0/2758162619 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0d50142f10 0x7f0d501420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d55699640 1 -- 172.21.15.103:0/2758162619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d5013f2a0 msgr2=0x7f0d50141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d55699640 1 --2- 172.21.15.103:0/2758162619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d5013f2a0 0x7f0d50141990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d55699640 1 -- 172.21.15.103:0/2758162619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5014c230 con 0x7f0d50148410 2024-08-19T13:19:09.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d54e98640 1 --2- 172.21.15.103:0/2758162619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d5013f2a0 0x7f0d50141990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:09.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.188+0000 7f0d4ffff640 1 --2- 172.21.15.103:0/2758162619 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0d50142f10 0x7f0d501420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:09.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.189+0000 7f0d55699640 1 --2- 172.21.15.103:0/2758162619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d50148410 0x7f0d50142620 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0d500b3110 tx=0x7f0d48004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:09.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.189+0000 7f0d4d7fa640 1 -- 172.21.15.103:0/2758162619 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48040070 con 0x7f0d50148410 2024-08-19T13:19:09.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.189+0000 7f0d4d7fa640 1 -- 172.21.15.103:0/2758162619 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d48031c00 con 0x7f0d50148410 2024-08-19T13:19:09.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.189+0000 7f0d570fc640 1 -- 172.21.15.103:0/2758162619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d50133410 con 0x7f0d50148410 2024-08-19T13:19:09.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.189+0000 7f0d4d7fa640 1 -- 172.21.15.103:0/2758162619 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48009450 con 0x7f0d50148410 2024-08-19T13:19:09.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.189+0000 7f0d570fc640 1 -- 172.21.15.103:0/2758162619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5014b040 con 0x7f0d50148410 2024-08-19T13:19:09.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.190+0000 7f0d570fc640 1 -- 172.21.15.103:0/2758162619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d50143860 con 0x7f0d50148410 2024-08-19T13:19:09.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.190+0000 7f0d4d7fa640 1 -- 172.21.15.103:0/2758162619 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d48039040 con 0x7f0d50148410 2024-08-19T13:19:09.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.191+0000 7f0d4d7fa640 1 --2- 172.21.15.103:0/2758162619 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f0d3406be30 0x7f0d3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.191+0000 7f0d4d7fa640 1 -- 172.21.15.103:0/2758162619 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7f0d480ad0a0 con 0x7f0d50148410 2024-08-19T13:19:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.191+0000 7f0d54e98640 1 --2- 172.21.15.103:0/2758162619 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f0d3406be30 0x7f0d3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.191+0000 7f0d4d7fa640 1 --2- 172.21.15.103:0/2758162619 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x7f0d34072ff0 0x7f0d34075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.191+0000 7f0d4d7fa640 1 -- 172.21.15.103:0/2758162619 --> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d480078e0 con 0x7f0d34072ff0 2024-08-19T13:19:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.191+0000 7f0d4d7fa640 1 -- 172.21.15.103:0/2758162619 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0d480ad530 con 0x7f0d50148410 2024-08-19T13:19:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.191+0000 7f0d4ffff640 1 --2- 172.21.15.103:0/2758162619 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x7f0d34072ff0 0x7f0d34075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.191+0000 7f0d54e98640 1 --2- 172.21.15.103:0/2758162619 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f0d3406be30 0x7f0d3406e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d44001ad0 tx=0x7f0d44009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.192+0000 7f0d4ffff640 1 --2- 172.21.15.103:0/2758162619 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x7f0d34072ff0 0x7f0d34075410 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.192+0000 7f0d4d7fa640 1 -- 172.21.15.103:0/2758162619 <== osd.5 v2:172.21.15.122:6816/2112974081 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0d480078e0 con 0x7f0d34072ff0 2024-08-19T13:19:09.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.206+0000 7f0d570fc640 1 -- 172.21.15.103:0/2758162619 --> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0d501336f0 con 0x7f0d34072ff0 2024-08-19T13:19:09.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.207+0000 7f0d4d7fa640 1 -- 172.21.15.103:0/2758162619 <== osd.5 v2:172.21.15.122:6816/2112974081 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0d501336f0 con 0x7f0d34072ff0 2024-08-19T13:19:09.207 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T13:19:09.208 INFO:teuthology.orchestra.run.smithi103.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-19T13:19:09.208 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T13:19:09.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.207+0000 7f0d570fc640 1 -- 172.21.15.103:0/2758162619 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x7f0d34072ff0 msgr2=0x7f0d34075410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.207+0000 7f0d570fc640 1 --2- 172.21.15.103:0/2758162619 >> [v2:172.21.15.122:6816/2112974081,v1:172.21.15.122:6817/2112974081] conn(0x7f0d34072ff0 0x7f0d34075410 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.207+0000 7f0d570fc640 1 -- 172.21.15.103:0/2758162619 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f0d3406be30 msgr2=0x7f0d3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.207+0000 7f0d570fc640 1 --2- 172.21.15.103:0/2758162619 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f0d3406be30 0x7f0d3406e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d44001ad0 tx=0x7f0d44009040 comp rx=0 tx=0).stop 2024-08-19T13:19:09.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.207+0000 7f0d570fc640 1 -- 172.21.15.103:0/2758162619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d50148410 msgr2=0x7f0d50142620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.207+0000 7f0d570fc640 1 --2- 172.21.15.103:0/2758162619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d50148410 0x7f0d50142620 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0d500b3110 tx=0x7f0d48004a40 comp rx=0 tx=0).stop 2024-08-19T13:19:09.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.207+0000 7f0d55699640 1 -- 172.21.15.103:0/2758162619 reap_dead start 2024-08-19T13:19:09.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.207+0000 7f0d570fc640 1 -- 172.21.15.103:0/2758162619 shutdown_connections 2024-08-19T13:19:09.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.207+0000 7f0d570fc640 1 -- 172.21.15.103:0/2758162619 >> 172.21.15.103:0/2758162619 conn(0x7f0d50067b10 msgr2=0x7f0d50070e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:09.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.207+0000 7f0d570fc640 1 -- 172.21.15.103:0/2758162619 shutdown_connections 2024-08-19T13:19:09.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.207+0000 7f0d570fc640 1 -- 172.21.15.103:0/2758162619 wait complete. 2024-08-19T13:19:09.220 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-08-19T13:19:09.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.331+0000 7f5a7821e640 1 Processor -- start 2024-08-19T13:19:09.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.331+0000 7f5a7821e640 1 -- start start 2024-08-19T13:19:09.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.332+0000 7f5a7821e640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f5a70137870 0x7f5a70141730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.332+0000 7f5a7821e640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a700a9840 con 0x7f5a70071be0 2024-08-19T13:19:09.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.332+0000 7f5a7821e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a70072bd0 con 0x7f5a70137490 2024-08-19T13:19:09.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.332+0000 7f5a7821e640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f5a700b28a0 con 0x7f5a70137870 2024-08-19T13:19:09.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.332+0000 7f5a757b9640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f5a70137870 0x7f5a70141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.332+0000 7f5a757b9640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f5a70137870 0x7f5a70141730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52722/0 (socket says 172.21.15.103:52722) 2024-08-19T13:19:09.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.332+0000 7f5a757b9640 1 -- 172.21.15.103:0/2580617024 learned_addr learned my addr 172.21.15.103:0/2580617024 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:09.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.332+0000 7f5a757b9640 1 -- 172.21.15.103:0/2580617024 >> v1:172.21.15.122:6789/0 conn(0x7f5a70137490 legacy=0x7f5a700b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.332+0000 7f5a757b9640 1 -- 172.21.15.103:0/2580617024 >> v1:172.21.15.103:6789/0 conn(0x7f5a70071be0 legacy=0x7f5a70072090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.332+0000 7f5a74fb8640 1 -- 172.21.15.103:0/2580617024 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2781512530 0 0) 0x7f5a70072bd0 con 0x7f5a70137490 2024-08-19T13:19:09.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.332+0000 7f5a757b9640 1 -- 172.21.15.103:0/2580617024 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a70141210 con 0x7f5a70137870 2024-08-19T13:19:09.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.333+0000 7f5a757b9640 1 --2- 172.21.15.103:0/2580617024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a70137870 0x7f5a70141730 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009870 tx=0x7f5a6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f80dfe26dd2f71f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:09.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.333+0000 7f5a74fb8640 1 -- 172.21.15.103:0/2580617024 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60037070 con 0x7f5a70137870 2024-08-19T13:19:09.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.333+0000 7f5a74fb8640 1 -- 172.21.15.103:0/2580617024 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a60032440 con 0x7f5a70137870 2024-08-19T13:19:09.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.333+0000 7f5a74fb8640 1 -- 172.21.15.103:0/2580617024 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60032810 con 0x7f5a70137870 2024-08-19T13:19:09.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.334+0000 7f5a7821e640 1 -- 172.21.15.103:0/2580617024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a70137870 msgr2=0x7f5a70141730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.334+0000 7f5a7821e640 1 --2- 172.21.15.103:0/2580617024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a70137870 0x7f5a70141730 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009870 tx=0x7f5a6002a1e0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.334+0000 7f5a7821e640 1 -- 172.21.15.103:0/2580617024 shutdown_connections 2024-08-19T13:19:09.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.334+0000 7f5a7821e640 1 --2- 172.21.15.103:0/2580617024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a70137870 0x7f5a70141730 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.334+0000 7f5a7821e640 1 -- 172.21.15.103:0/2580617024 >> 172.21.15.103:0/2580617024 conn(0x7f5a700b6b10 msgr2=0x7f5a700b6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:09.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.334+0000 7f5a7821e640 1 -- 172.21.15.103:0/2580617024 shutdown_connections 2024-08-19T13:19:09.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.334+0000 7f5a7821e640 1 -- 172.21.15.103:0/2580617024 wait complete. 2024-08-19T13:19:09.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.335+0000 7f5a7821e640 1 Processor -- start 2024-08-19T13:19:09.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.335+0000 7f5a7821e640 1 -- start start 2024-08-19T13:19:09.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a7821e640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a70071be0 0x7f5a7013d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a7821e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a70137490 0x7f5a701db460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a7821e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a70137870 0x7f5a701db9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a7821e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5a700b19b0 con 0x7f5a70137870 2024-08-19T13:19:09.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a7821e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5a700b1830 con 0x7f5a70137490 2024-08-19T13:19:09.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a7821e640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f5a7014c4f0 con 0x7f5a70071be0 2024-08-19T13:19:09.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a75fba640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a70071be0 0x7f5a7013d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a767bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a70137870 0x7f5a701db9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a75fba640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a70071be0 0x7f5a7013d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41654/0 (socket says 172.21.15.103:41654) 2024-08-19T13:19:09.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a75fba640 1 -- 172.21.15.103:0/2942690731 learned_addr learned my addr 172.21.15.103:0/2942690731 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:09.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a757b9640 1 --2- 172.21.15.103:0/2942690731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a70137490 0x7f5a701db460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a767bb640 1 -- 172.21.15.103:0/2942690731 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a70071be0 msgr2=0x7f5a7013d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a767bb640 1 --2- 172.21.15.103:0/2942690731 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a70071be0 0x7f5a7013d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a767bb640 1 -- 172.21.15.103:0/2942690731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a70137490 msgr2=0x7f5a701db460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a767bb640 1 --2- 172.21.15.103:0/2942690731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a70137490 0x7f5a701db460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a767bb640 1 -- 172.21.15.103:0/2942690731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a70136840 con 0x7f5a70137870 2024-08-19T13:19:09.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a757b9640 1 --2- 172.21.15.103:0/2942690731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a70137490 0x7f5a701db460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:09.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a767bb640 1 --2- 172.21.15.103:0/2942690731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a70137870 0x7f5a701db9a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800c6c0 tx=0x7f5a6800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:09.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a667fc640 1 -- 172.21.15.103:0/2942690731 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68003b70 con 0x7f5a70137870 2024-08-19T13:19:09.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a667fc640 1 -- 172.21.15.103:0/2942690731 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a68003d10 con 0x7f5a70137870 2024-08-19T13:19:09.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a7821e640 1 -- 172.21.15.103:0/2942690731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a70136b30 con 0x7f5a70137870 2024-08-19T13:19:09.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a667fc640 1 -- 172.21.15.103:0/2942690731 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a680052f0 con 0x7f5a70137870 2024-08-19T13:19:09.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.336+0000 7f5a7821e640 1 -- 172.21.15.103:0/2942690731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a701370a0 con 0x7f5a70137870 2024-08-19T13:19:09.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.337+0000 7f5a7821e640 1 -- 172.21.15.103:0/2942690731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a70141730 con 0x7f5a70137870 2024-08-19T13:19:09.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.338+0000 7f5a667fc640 1 -- 172.21.15.103:0/2942690731 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a68005490 con 0x7f5a70137870 2024-08-19T13:19:09.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.338+0000 7f5a667fc640 1 --2- 172.21.15.103:0/2942690731 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f5a5406be60 0x7f5a5406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.339+0000 7f5a75fba640 1 --2- 172.21.15.103:0/2942690731 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f5a5406be60 0x7f5a5406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.339+0000 7f5a667fc640 1 -- 172.21.15.103:0/2942690731 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7f5a6808a9e0 con 0x7f5a70137870 2024-08-19T13:19:09.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.339+0000 7f5a75fba640 1 --2- 172.21.15.103:0/2942690731 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f5a5406be60 0x7f5a5406e320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0018c0 tx=0x7f5a6c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:09.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.343+0000 7f5a667fc640 1 -- 172.21.15.103:0/2942690731 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f5a68058e10 con 0x7f5a70137870 2024-08-19T13:19:09.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.511+0000 7f5a7821e640 1 -- 172.21.15.103:0/2942690731 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f5a701372d0 con 0x7f5a5406be60 2024-08-19T13:19:09.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.512+0000 7f5a667fc640 1 -- 172.21.15.103:0/2942690731 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5a6c00f7d0 con 0x7f5a5406be60 2024-08-19T13:19:09.513 INFO:teuthology.orchestra.run.smithi103.stderr:instructed osd(s) 5 to deep-scrub 2024-08-19T13:19:09.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.514+0000 7f5a7821e640 1 -- 172.21.15.103:0/2942690731 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f5a5406be60 msgr2=0x7f5a5406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.514+0000 7f5a7821e640 1 --2- 172.21.15.103:0/2942690731 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f5a5406be60 0x7f5a5406e320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0018c0 tx=0x7f5a6c009040 comp rx=0 tx=0).stop 2024-08-19T13:19:09.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.514+0000 7f5a7821e640 1 -- 172.21.15.103:0/2942690731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a70137870 msgr2=0x7f5a701db9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.514+0000 7f5a7821e640 1 --2- 172.21.15.103:0/2942690731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a70137870 0x7f5a701db9a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800c6c0 tx=0x7f5a6800cb80 comp rx=0 tx=0).stop 2024-08-19T13:19:09.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.514+0000 7f5a7821e640 1 -- 172.21.15.103:0/2942690731 shutdown_connections 2024-08-19T13:19:09.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.514+0000 7f5a7821e640 1 --2- 172.21.15.103:0/2942690731 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f5a5406be60 0x7f5a5406e320 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.514+0000 7f5a7821e640 1 --2- 172.21.15.103:0/2942690731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a70137870 0x7f5a701db9a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.514+0000 7f5a7821e640 1 --2- 172.21.15.103:0/2942690731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a70137490 0x7f5a701db460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.514+0000 7f5a7821e640 1 --2- 172.21.15.103:0/2942690731 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a70071be0 0x7f5a7013d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.514+0000 7f5a7821e640 1 -- 172.21.15.103:0/2942690731 >> 172.21.15.103:0/2942690731 conn(0x7f5a700b6b10 msgr2=0x7f5a70072090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:09.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.514+0000 7f5a7821e640 1 -- 172.21.15.103:0/2942690731 shutdown_connections 2024-08-19T13:19:09.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.514+0000 7f5a7821e640 1 -- 172.21.15.103:0/2942690731 wait complete. 2024-08-19T13:19:09.525 INFO:tasks.ceph:Scrubbing osd.6 2024-08-19T13:19:09.525 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-08-19T13:19:09.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.626+0000 7fa0c7e42640 1 Processor -- start 2024-08-19T13:19:09.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.626+0000 7fa0c7e42640 1 -- start start 2024-08-19T13:19:09.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.626+0000 7fa0c7e42640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa0c0071cc0 0x7fa0c01436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.626+0000 7fa0c7e42640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa0c00706b0 0x7fa0c0142b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.626+0000 7fa0c7e42640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0c00adbb0 con 0x7fa0c00720a0 2024-08-19T13:19:09.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.627+0000 7fa0c7e42640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fa0c00724f0 con 0x7fa0c0071cc0 2024-08-19T13:19:09.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.627+0000 7fa0c7e42640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fa0c01431d0 con 0x7fa0c00706b0 2024-08-19T13:19:09.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.627+0000 7fa0c5bde640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa0c0071cc0 0x7fa0c01436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.627+0000 7fa0c5bde640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa0c0071cc0 0x7fa0c01436f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52744/0 (socket says 172.21.15.103:52744) 2024-08-19T13:19:09.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.627+0000 7fa0c5bde640 1 -- 172.21.15.103:0/609896905 learned_addr learned my addr 172.21.15.103:0/609896905 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:09.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.627+0000 7fa0c63df640 1 --2- 172.21.15.103:0/609896905 >> v2:172.21.15.122:3300/0 conn(0x7fa0c00706b0 0x7fa0c0142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.627+0000 7fa0c5bde640 1 -- 172.21.15.103:0/609896905 >> v2:172.21.15.122:3300/0 conn(0x7fa0c00706b0 msgr2=0x7fa0c0142b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:09.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.627+0000 7fa0c5bde640 1 --2- 172.21.15.103:0/609896905 >> v2:172.21.15.122:3300/0 conn(0x7fa0c00706b0 0x7fa0c0142b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.627+0000 7fa0c5bde640 1 -- 172.21.15.103:0/609896905 >> v1:172.21.15.103:6789/0 conn(0x7fa0c00720a0 legacy=0x7fa0c006ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.627+0000 7fa0c5bde640 1 -- 172.21.15.103:0/609896905 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c013fb00 con 0x7fa0c0071cc0 2024-08-19T13:19:09.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.627+0000 7fa0c63df640 1 --2- 172.21.15.103:0/609896905 >> v2:172.21.15.122:3300/0 conn(0x7fa0c00706b0 0x7fa0c0142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T13:19:09.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.628+0000 7fa0c5bde640 1 --2- 172.21.15.103:0/609896905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0c0071cc0 0x7fa0c01436f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa0c006ef20 tx=0x7fa0b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f94168ba866011 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:09.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.628+0000 7fa0c4bdc640 1 -- 172.21.15.103:0/609896905 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0037070 con 0x7fa0c0071cc0 2024-08-19T13:19:09.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.628+0000 7fa0c4bdc640 1 -- 172.21.15.103:0/609896905 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0b0032440 con 0x7fa0c0071cc0 2024-08-19T13:19:09.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.628+0000 7fa0c4bdc640 1 -- 172.21.15.103:0/609896905 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0032810 con 0x7fa0c0071cc0 2024-08-19T13:19:09.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.628+0000 7fa0c7e42640 1 -- 172.21.15.103:0/609896905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0c0071cc0 msgr2=0x7fa0c01436f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.628+0000 7fa0c7e42640 1 --2- 172.21.15.103:0/609896905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0c0071cc0 0x7fa0c01436f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa0c006ef20 tx=0x7fa0b002a1e0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.628+0000 7fa0c7e42640 1 -- 172.21.15.103:0/609896905 shutdown_connections 2024-08-19T13:19:09.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.628+0000 7fa0c7e42640 1 --2- 172.21.15.103:0/609896905 >> v2:172.21.15.122:3300/0 conn(0x7fa0c00706b0 0x7fa0c0142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.628+0000 7fa0c7e42640 1 --2- 172.21.15.103:0/609896905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0c0071cc0 0x7fa0c01436f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.628+0000 7fa0c7e42640 1 -- 172.21.15.103:0/609896905 >> 172.21.15.103:0/609896905 conn(0x7fa0c00b5370 msgr2=0x7fa0c00b5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:09.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.628+0000 7fa0c7e42640 1 -- 172.21.15.103:0/609896905 shutdown_connections 2024-08-19T13:19:09.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.628+0000 7fa0c7e42640 1 -- 172.21.15.103:0/609896905 wait complete. 2024-08-19T13:19:09.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.629+0000 7fa0c7e42640 1 Processor -- start 2024-08-19T13:19:09.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.629+0000 7fa0c7e42640 1 -- start start 2024-08-19T13:19:09.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c7e42640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0c0071cc0 0x7fa0c00b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c7e42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0c00720a0 0x7fa0c00af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c7e42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0c0138a20 0x7fa0c0138ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c7e42640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c0140290 con 0x7fa0c00720a0 2024-08-19T13:19:09.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c7e42640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c0140110 con 0x7fa0c0138a20 2024-08-19T13:19:09.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c7e42640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa0c0140410 con 0x7fa0c0071cc0 2024-08-19T13:19:09.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c53dd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0c00720a0 0x7fa0c00af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c5bde640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0c0071cc0 0x7fa0c00b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c53dd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0c00720a0 0x7fa0c00af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52758/0 (socket says 172.21.15.103:52758) 2024-08-19T13:19:09.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c53dd640 1 -- 172.21.15.103:0/3929474301 learned_addr learned my addr 172.21.15.103:0/3929474301 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:09.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c63df640 1 --2- 172.21.15.103:0/3929474301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0c0138a20 0x7fa0c0138ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c53dd640 1 -- 172.21.15.103:0/3929474301 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0c0071cc0 msgr2=0x7fa0c00b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c53dd640 1 --2- 172.21.15.103:0/3929474301 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0c0071cc0 0x7fa0c00b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c53dd640 1 -- 172.21.15.103:0/3929474301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0c0138a20 msgr2=0x7fa0c0138ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c53dd640 1 --2- 172.21.15.103:0/3929474301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0c0138a20 0x7fa0c0138ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c53dd640 1 -- 172.21.15.103:0/3929474301 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c00aea10 con 0x7fa0c00720a0 2024-08-19T13:19:09.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.630+0000 7fa0c5bde640 1 --2- 172.21.15.103:0/3929474301 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0c0071cc0 0x7fa0c00b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:09.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.631+0000 7fa0c53dd640 1 --2- 172.21.15.103:0/3929474301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0c00720a0 0x7fa0c00af4e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00c3f0 tx=0x7fa0bc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:09.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.631+0000 7fa0b67fc640 1 -- 172.21.15.103:0/3929474301 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0052d0 con 0x7fa0c00720a0 2024-08-19T13:19:09.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.631+0000 7fa0b67fc640 1 -- 172.21.15.103:0/3929474301 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0bc005490 con 0x7fa0c00720a0 2024-08-19T13:19:09.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.631+0000 7fa0c7e42640 1 -- 172.21.15.103:0/3929474301 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c0139410 con 0x7fa0c00720a0 2024-08-19T13:19:09.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.631+0000 7fa0b67fc640 1 -- 172.21.15.103:0/3929474301 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc007260 con 0x7fa0c00720a0 2024-08-19T13:19:09.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.631+0000 7fa0c7e42640 1 -- 172.21.15.103:0/3929474301 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c00ae630 con 0x7fa0c00720a0 2024-08-19T13:19:09.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.632+0000 7fa0c7e42640 1 -- 172.21.15.103:0/3929474301 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa0c0142b00 con 0x7fa0c00720a0 2024-08-19T13:19:09.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.632+0000 7fa0b67fc640 1 -- 172.21.15.103:0/3929474301 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa0bc007400 con 0x7fa0c00720a0 2024-08-19T13:19:09.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.633+0000 7fa0b67fc640 1 --2- 172.21.15.103:0/3929474301 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa0a406be30 0x7fa0a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.633+0000 7fa0b67fc640 1 -- 172.21.15.103:0/3929474301 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7fa0bc08a1d0 con 0x7fa0c00720a0 2024-08-19T13:19:09.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.633+0000 7fa0c5bde640 1 --2- 172.21.15.103:0/3929474301 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa0a406be30 0x7fa0a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.633+0000 7fa0b67fc640 1 --2- 172.21.15.103:0/3929474301 >> [v2:172.21.15.122:6808/2033312647,v1:172.21.15.122:6809/2033312647] conn(0x7fa0a4072ff0 0x7fa0a4075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.633+0000 7fa0b67fc640 1 -- 172.21.15.103:0/3929474301 --> [v2:172.21.15.122:6808/2033312647,v1:172.21.15.122:6809/2033312647] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa0bc009e50 con 0x7fa0a4072ff0 2024-08-19T13:19:09.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.633+0000 7fa0b67fc640 1 -- 172.21.15.103:0/3929474301 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa0bc091050 con 0x7fa0c00720a0 2024-08-19T13:19:09.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.633+0000 7fa0c63df640 1 --2- 172.21.15.103:0/3929474301 >> [v2:172.21.15.122:6808/2033312647,v1:172.21.15.122:6809/2033312647] conn(0x7fa0a4072ff0 0x7fa0a4075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.633+0000 7fa0c5bde640 1 --2- 172.21.15.103:0/3929474301 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa0a406be30 0x7fa0a406e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0034000 tx=0x7fa0b002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:09.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.633+0000 7fa0c63df640 1 --2- 172.21.15.103:0/3929474301 >> [v2:172.21.15.122:6808/2033312647,v1:172.21.15.122:6809/2033312647] conn(0x7fa0a4072ff0 0x7fa0a4075410 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:09.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.634+0000 7fa0b67fc640 1 -- 172.21.15.103:0/3929474301 <== osd.6 v2:172.21.15.122:6808/2033312647 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa0bc009e50 con 0x7fa0a4072ff0 2024-08-19T13:19:09.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.646+0000 7fa0c7e42640 1 -- 172.21.15.103:0/3929474301 --> [v2:172.21.15.122:6808/2033312647,v1:172.21.15.122:6809/2033312647] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa0c00b6dc0 con 0x7fa0a4072ff0 2024-08-19T13:19:09.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.647+0000 7fa0b67fc640 1 -- 172.21.15.103:0/3929474301 <== osd.6 v2:172.21.15.122:6808/2033312647 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa0c00b6dc0 con 0x7fa0a4072ff0 2024-08-19T13:19:09.647 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T13:19:09.647 INFO:teuthology.orchestra.run.smithi103.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-19T13:19:09.647 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T13:19:09.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.647+0000 7fa0c7e42640 1 -- 172.21.15.103:0/3929474301 >> [v2:172.21.15.122:6808/2033312647,v1:172.21.15.122:6809/2033312647] conn(0x7fa0a4072ff0 msgr2=0x7fa0a4075410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.647+0000 7fa0c7e42640 1 --2- 172.21.15.103:0/3929474301 >> [v2:172.21.15.122:6808/2033312647,v1:172.21.15.122:6809/2033312647] conn(0x7fa0a4072ff0 0x7fa0a4075410 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.647+0000 7fa0c7e42640 1 -- 172.21.15.103:0/3929474301 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa0a406be30 msgr2=0x7fa0a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.647+0000 7fa0c7e42640 1 --2- 172.21.15.103:0/3929474301 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fa0a406be30 0x7fa0a406e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0034000 tx=0x7fa0b002e000 comp rx=0 tx=0).stop 2024-08-19T13:19:09.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.647+0000 7fa0c7e42640 1 -- 172.21.15.103:0/3929474301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0c00720a0 msgr2=0x7fa0c00af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.647+0000 7fa0c7e42640 1 --2- 172.21.15.103:0/3929474301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0c00720a0 0x7fa0c00af4e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00c3f0 tx=0x7fa0bc00c8b0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.647+0000 7fa0c63df640 1 -- 172.21.15.103:0/3929474301 reap_dead start 2024-08-19T13:19:09.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.647+0000 7fa0c7e42640 1 -- 172.21.15.103:0/3929474301 shutdown_connections 2024-08-19T13:19:09.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.647+0000 7fa0c7e42640 1 -- 172.21.15.103:0/3929474301 >> 172.21.15.103:0/3929474301 conn(0x7fa0c00b5370 msgr2=0x7fa0c0072bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:09.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.647+0000 7fa0c7e42640 1 -- 172.21.15.103:0/3929474301 shutdown_connections 2024-08-19T13:19:09.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.647+0000 7fa0c7e42640 1 -- 172.21.15.103:0/3929474301 wait complete. 2024-08-19T13:19:09.658 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-08-19T13:19:09.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.758+0000 7f3749463640 1 Processor -- start 2024-08-19T13:19:09.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.758+0000 7f3749463640 1 -- start start 2024-08-19T13:19:09.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.759+0000 7f3749463640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f37440b6000 0x7f37440b1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.759+0000 7f3749463640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f37440b0420 0x7f37441444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.759+0000 7f3749463640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37441409d0 0x7f3744144a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.759+0000 7f3749463640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f37440b63e0 con 0x7f37440b0420 2024-08-19T13:19:09.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.759+0000 7f3749463640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f37440b1ef0 con 0x7f37440b6000 2024-08-19T13:19:09.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.759+0000 7f3749463640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f374413fc90 con 0x7f37441409d0 2024-08-19T13:19:09.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.759+0000 7f37427fc640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f37440b0420 0x7f37441444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.759+0000 7f37437fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37441409d0 0x7f3744144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.759+0000 7f37427fc640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f37440b0420 0x7f37441444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52774/0 (socket says 172.21.15.103:52774) 2024-08-19T13:19:09.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.759+0000 7f37427fc640 1 -- 172.21.15.103:0/12890235 learned_addr learned my addr 172.21.15.103:0/12890235 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:09.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.759+0000 7f3742ffd640 1 --2- 172.21.15.103:0/12890235 >> v2:172.21.15.122:3300/0 conn(0x7f37440b6000 0x7f37440b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.760+0000 7f37437fe640 1 -- 172.21.15.103:0/12890235 >> v2:172.21.15.122:3300/0 conn(0x7f37440b6000 msgr2=0x7f37440b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:09.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.760+0000 7f37437fe640 1 --2- 172.21.15.103:0/12890235 >> v2:172.21.15.122:3300/0 conn(0x7f37440b6000 0x7f37440b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.760+0000 7f37437fe640 1 -- 172.21.15.103:0/12890235 >> v2:172.21.15.103:3300/0 conn(0x7f37440b0420 msgr2=0x7f37441444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:09.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.760+0000 7f37437fe640 1 --2- 172.21.15.103:0/12890235 >> v2:172.21.15.103:3300/0 conn(0x7f37440b0420 0x7f37441444d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.760+0000 7f37437fe640 1 -- 172.21.15.103:0/12890235 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374413fe40 con 0x7f37441409d0 2024-08-19T13:19:09.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.760+0000 7f37427fc640 1 --2- 172.21.15.103:0/12890235 >> v2:172.21.15.103:3300/0 conn(0x7f37440b0420 0x7f37441444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:09.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.760+0000 7f37437fe640 1 --2- 172.21.15.103:0/12890235 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37441409d0 0x7f3744144a10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f37440729b0 tx=0x7f373c02a540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77d898fffae8fd2d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:09.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.760+0000 7f3741ffb640 1 -- 172.21.15.103:0/12890235 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c007740 con 0x7f37441409d0 2024-08-19T13:19:09.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.760+0000 7f3741ffb640 1 -- 172.21.15.103:0/12890235 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f373c0078e0 con 0x7f37441409d0 2024-08-19T13:19:09.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.760+0000 7f3741ffb640 1 -- 172.21.15.103:0/12890235 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c007cb0 con 0x7f37441409d0 2024-08-19T13:19:09.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.761+0000 7f3749463640 1 -- 172.21.15.103:0/12890235 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37441409d0 msgr2=0x7f3744144a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.761+0000 7f3749463640 1 --2- 172.21.15.103:0/12890235 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37441409d0 0x7f3744144a10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f37440729b0 tx=0x7f373c02a540 comp rx=0 tx=0).stop 2024-08-19T13:19:09.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.761+0000 7f3749463640 1 -- 172.21.15.103:0/12890235 shutdown_connections 2024-08-19T13:19:09.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.761+0000 7f3749463640 1 --2- 172.21.15.103:0/12890235 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37441409d0 0x7f3744144a10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.761+0000 7f3749463640 1 --2- 172.21.15.103:0/12890235 >> v2:172.21.15.103:3300/0 conn(0x7f37440b0420 0x7f37441444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.761+0000 7f3749463640 1 --2- 172.21.15.103:0/12890235 >> v2:172.21.15.122:3300/0 conn(0x7f37440b6000 0x7f37440b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.761+0000 7f3749463640 1 -- 172.21.15.103:0/12890235 >> 172.21.15.103:0/12890235 conn(0x7f3744067d20 msgr2=0x7f37440b5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:09.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.761+0000 7f3749463640 1 -- 172.21.15.103:0/12890235 shutdown_connections 2024-08-19T13:19:09.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.761+0000 7f3749463640 1 -- 172.21.15.103:0/12890235 wait complete. 2024-08-19T13:19:09.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.762+0000 7f3749463640 1 Processor -- start 2024-08-19T13:19:09.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.762+0000 7f3749463640 1 -- start start 2024-08-19T13:19:09.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f3749463640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37440b0420 0x7f37440aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f3749463640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37440b6000 0x7f3744071d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f3749463640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37441409d0 0x7f37441db590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f3749463640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37441405d0 con 0x7f37440b0420 2024-08-19T13:19:09.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f3749463640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3744140450 con 0x7f37440b6000 2024-08-19T13:19:09.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f3749463640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3744140750 con 0x7f37441409d0 2024-08-19T13:19:09.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f3742ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37440b0420 0x7f37440aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f37427fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37440b6000 0x7f3744071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f37437fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37441409d0 0x7f37441db590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f3742ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37440b0420 0x7f37440aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52780/0 (socket says 172.21.15.103:52780) 2024-08-19T13:19:09.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f3742ffd640 1 -- 172.21.15.103:0/890646033 learned_addr learned my addr 172.21.15.103:0/890646033 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:09.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f37437fe640 1 -- 172.21.15.103:0/890646033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37440b6000 msgr2=0x7f3744071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f37437fe640 1 --2- 172.21.15.103:0/890646033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37440b6000 0x7f3744071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f37437fe640 1 -- 172.21.15.103:0/890646033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37440b0420 msgr2=0x7f37440aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f37437fe640 1 --2- 172.21.15.103:0/890646033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37440b0420 0x7f37440aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f37437fe640 1 -- 172.21.15.103:0/890646033 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374411dc60 con 0x7f37441409d0 2024-08-19T13:19:09.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f3742ffd640 1 --2- 172.21.15.103:0/890646033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37440b0420 0x7f37440aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:09.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.763+0000 7f37427fc640 1 --2- 172.21.15.103:0/890646033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37440b6000 0x7f3744071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:09.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.764+0000 7f37437fe640 1 --2- 172.21.15.103:0/890646033 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37441409d0 0x7f37441db590 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f374413fe10 tx=0x7f373c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:09.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.764+0000 7f372b7fe640 1 -- 172.21.15.103:0/890646033 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c03f070 con 0x7f37441409d0 2024-08-19T13:19:09.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.764+0000 7f372b7fe640 1 -- 172.21.15.103:0/890646033 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f373c031bc0 con 0x7f37441409d0 2024-08-19T13:19:09.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.764+0000 7f3749463640 1 -- 172.21.15.103:0/890646033 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374411eee0 con 0x7f37441409d0 2024-08-19T13:19:09.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.764+0000 7f372b7fe640 1 -- 172.21.15.103:0/890646033 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c009450 con 0x7f37441409d0 2024-08-19T13:19:09.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.764+0000 7f3749463640 1 -- 172.21.15.103:0/890646033 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f374411d440 con 0x7f37441409d0 2024-08-19T13:19:09.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.765+0000 7f3749463640 1 -- 172.21.15.103:0/890646033 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37440b1010 con 0x7f37441409d0 2024-08-19T13:19:09.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.765+0000 7f372b7fe640 1 -- 172.21.15.103:0/890646033 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f373c044460 con 0x7f37441409d0 2024-08-19T13:19:09.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.766+0000 7f372b7fe640 1 --2- 172.21.15.103:0/890646033 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f373006be30 0x7f373006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:09.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.766+0000 7f372b7fe640 1 -- 172.21.15.103:0/890646033 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7f373c0ac330 con 0x7f37441409d0 2024-08-19T13:19:09.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.766+0000 7f3742ffd640 1 --2- 172.21.15.103:0/890646033 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f373006be30 0x7f373006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:09.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.766+0000 7f3742ffd640 1 --2- 172.21.15.103:0/890646033 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f373006be30 0x7f373006e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3734009f70 tx=0x7f3734009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:09.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.770+0000 7f372b7fe640 1 -- 172.21.15.103:0/890646033 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f373c07a760 con 0x7f37441409d0 2024-08-19T13:19:09.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.965+0000 7f3749463640 1 -- 172.21.15.103:0/890646033 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f3744075a00 con 0x7f373006be30 2024-08-19T13:19:09.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.965+0000 7f372b7fe640 1 -- 172.21.15.103:0/890646033 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3734009ca0 con 0x7f373006be30 2024-08-19T13:19:09.966 INFO:teuthology.orchestra.run.smithi103.stderr:instructed osd(s) 6 to deep-scrub 2024-08-19T13:19:09.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.967+0000 7f3749463640 1 -- 172.21.15.103:0/890646033 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f373006be30 msgr2=0x7f373006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.967+0000 7f3749463640 1 --2- 172.21.15.103:0/890646033 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f373006be30 0x7f373006e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3734009f70 tx=0x7f3734009190 comp rx=0 tx=0).stop 2024-08-19T13:19:09.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.967+0000 7f3749463640 1 -- 172.21.15.103:0/890646033 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37441409d0 msgr2=0x7f37441db590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:09.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.967+0000 7f3749463640 1 --2- 172.21.15.103:0/890646033 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37441409d0 0x7f37441db590 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f374413fe10 tx=0x7f373c004a40 comp rx=0 tx=0).stop 2024-08-19T13:19:09.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.967+0000 7f3749463640 1 -- 172.21.15.103:0/890646033 shutdown_connections 2024-08-19T13:19:09.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.967+0000 7f3749463640 1 --2- 172.21.15.103:0/890646033 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f373006be30 0x7f373006e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.967+0000 7f3749463640 1 --2- 172.21.15.103:0/890646033 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37441409d0 0x7f37441db590 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.967+0000 7f3749463640 1 --2- 172.21.15.103:0/890646033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37440b6000 0x7f3744071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.967+0000 7f3749463640 1 --2- 172.21.15.103:0/890646033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37440b0420 0x7f37440aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:09.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.967+0000 7f3749463640 1 -- 172.21.15.103:0/890646033 >> 172.21.15.103:0/890646033 conn(0x7f3744067d20 msgr2=0x7f3744142070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:09.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.968+0000 7f3749463640 1 -- 172.21.15.103:0/890646033 shutdown_connections 2024-08-19T13:19:09.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:09.968+0000 7f3749463640 1 -- 172.21.15.103:0/890646033 wait complete. 2024-08-19T13:19:09.982 INFO:tasks.ceph:Scrubbing osd.7 2024-08-19T13:19:09.982 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-08-19T13:19:10.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.110+0000 7f439c2ad640 1 Processor -- start 2024-08-19T13:19:10.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.110+0000 7f439c2ad640 1 -- start start 2024-08-19T13:19:10.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.111+0000 7f439c2ad640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43940a94c0 0x7f43941437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.111+0000 7f439c2ad640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f4394071080 0x7f43940714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.111+0000 7f439c2ad640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4394073de0 con 0x7f4394142ef0 2024-08-19T13:19:10.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.111+0000 7f439c2ad640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f439413f4d0 con 0x7f4394071080 2024-08-19T13:19:10.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.111+0000 7f439c2ad640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f439406e280 con 0x7f43940a94c0 2024-08-19T13:19:10.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.111+0000 7f439a84a640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f4394071080 0x7f43940714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.111+0000 7f4399848640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43940a94c0 0x7f43941437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.111+0000 7f439a84a640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f4394071080 0x7f43940714b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52796/0 (socket says 172.21.15.103:52796) 2024-08-19T13:19:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.111+0000 7f439a84a640 1 -- 172.21.15.103:0/2315845780 learned_addr learned my addr 172.21.15.103:0/2315845780 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.112+0000 7f4399848640 1 -- 172.21.15.103:0/2315845780 >> v2:172.21.15.103:3300/0 conn(0x7f4394071080 msgr2=0x7f43940714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.112+0000 7f4399848640 1 --2- 172.21.15.103:0/2315845780 >> v2:172.21.15.103:3300/0 conn(0x7f4394071080 0x7f43940714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.112+0000 7f4399848640 1 -- 172.21.15.103:0/2315845780 >> v1:172.21.15.103:6789/0 conn(0x7f4394142ef0 legacy=0x7f439413edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.112+0000 7f4399848640 1 -- 172.21.15.103:0/2315845780 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43941432d0 con 0x7f43940a94c0 2024-08-19T13:19:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.113+0000 7f4399848640 1 --2- 172.21.15.103:0/2315845780 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43940a94c0 0x7f43941437d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f438800b850 tx=0x7f438802a500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91279716b7c49070 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.113+0000 7f4399047640 1 -- 172.21.15.103:0/2315845780 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438802f710 con 0x7f43940a94c0 2024-08-19T13:19:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.113+0000 7f4399047640 1 -- 172.21.15.103:0/2315845780 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f438802f8b0 con 0x7f43940a94c0 2024-08-19T13:19:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.113+0000 7f4399047640 1 -- 172.21.15.103:0/2315845780 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438802fc80 con 0x7f43940a94c0 2024-08-19T13:19:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.114+0000 7f439c2ad640 1 -- 172.21.15.103:0/2315845780 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43940a94c0 msgr2=0x7f43941437d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.114+0000 7f439c2ad640 1 --2- 172.21.15.103:0/2315845780 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43940a94c0 0x7f43941437d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f438800b850 tx=0x7f438802a500 comp rx=0 tx=0).stop 2024-08-19T13:19:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.114+0000 7f439c2ad640 1 -- 172.21.15.103:0/2315845780 shutdown_connections 2024-08-19T13:19:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.114+0000 7f439c2ad640 1 --2- 172.21.15.103:0/2315845780 >> v2:172.21.15.103:3300/0 conn(0x7f4394071080 0x7f43940714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.114+0000 7f439c2ad640 1 --2- 172.21.15.103:0/2315845780 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43940a94c0 0x7f43941437d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.114+0000 7f439c2ad640 1 -- 172.21.15.103:0/2315845780 >> 172.21.15.103:0/2315845780 conn(0x7f43940b1f70 msgr2=0x7f43940b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.115+0000 7f439c2ad640 1 -- 172.21.15.103:0/2315845780 shutdown_connections 2024-08-19T13:19:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.115+0000 7f439c2ad640 1 -- 172.21.15.103:0/2315845780 wait complete. 2024-08-19T13:19:10.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.116+0000 7f439c2ad640 1 Processor -- start 2024-08-19T13:19:10.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.116+0000 7f439c2ad640 1 -- start start 2024-08-19T13:19:10.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.117+0000 7f439c2ad640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4394071080 0x7f439413e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.117+0000 7f439c2ad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4394142ef0 0x7f4394141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.117+0000 7f439c2ad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394136ea0 0x7f43941d7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.117+0000 7f439c2ad640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f439406c6f0 con 0x7f4394142ef0 2024-08-19T13:19:10.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.117+0000 7f439c2ad640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f439406c570 con 0x7f4394136ea0 2024-08-19T13:19:10.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.117+0000 7f439c2ad640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f439406c870 con 0x7f4394071080 2024-08-19T13:19:10.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.117+0000 7f439a049640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4394071080 0x7f439413e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.117+0000 7f439a049640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4394071080 0x7f439413e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41712/0 (socket says 172.21.15.103:41712) 2024-08-19T13:19:10.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.117+0000 7f439a049640 1 -- 172.21.15.103:0/2763009988 learned_addr learned my addr 172.21.15.103:0/2763009988 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:10.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.117+0000 7f4399848640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4394142ef0 0x7f4394141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.118+0000 7f439a84a640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394136ea0 0x7f43941d7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.118+0000 7f439a049640 1 -- 172.21.15.103:0/2763009988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394136ea0 msgr2=0x7f43941d7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.118+0000 7f439a049640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394136ea0 0x7f43941d7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.118+0000 7f439a049640 1 -- 172.21.15.103:0/2763009988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4394142ef0 msgr2=0x7f4394141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.118+0000 7f439a049640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4394142ef0 0x7f4394141e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.118+0000 7f439a049640 1 -- 172.21.15.103:0/2763009988 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439413dcc0 con 0x7f4394071080 2024-08-19T13:19:10.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.118+0000 7f4399848640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4394142ef0 0x7f4394141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:10.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.118+0000 7f439a049640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4394071080 0x7f439413e530 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f439000c3f0 tx=0x7f439000c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:10.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.119+0000 7f4386ffd640 1 -- 172.21.15.103:0/2763009988 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900052d0 con 0x7f4394071080 2024-08-19T13:19:10.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.119+0000 7f4386ffd640 1 -- 172.21.15.103:0/2763009988 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4390005490 con 0x7f4394071080 2024-08-19T13:19:10.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.119+0000 7f4386ffd640 1 -- 172.21.15.103:0/2763009988 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390007260 con 0x7f4394071080 2024-08-19T13:19:10.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.119+0000 7f439c2ad640 1 -- 172.21.15.103:0/2763009988 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4394136970 con 0x7f4394071080 2024-08-19T13:19:10.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.119+0000 7f439c2ad640 1 -- 172.21.15.103:0/2763009988 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f43941d7570 con 0x7f4394071080 2024-08-19T13:19:10.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.120+0000 7f439c2ad640 1 -- 172.21.15.103:0/2763009988 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43940714b0 con 0x7f4394071080 2024-08-19T13:19:10.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.121+0000 7f4386ffd640 1 -- 172.21.15.103:0/2763009988 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4390007400 con 0x7f4394071080 2024-08-19T13:19:10.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.122+0000 7f4386ffd640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f437806be60 0x7f437806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.122+0000 7f4386ffd640 1 -- 172.21.15.103:0/2763009988 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7f439008a270 con 0x7f4394071080 2024-08-19T13:19:10.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.122+0000 7f4399848640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f437806be60 0x7f437806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.122+0000 7f4386ffd640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.122:6824/3566117434,v1:172.21.15.122:6825/3566117434] conn(0x7f4378073020 0x7f4378075440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.122+0000 7f4386ffd640 1 -- 172.21.15.103:0/2763009988 --> [v2:172.21.15.122:6824/3566117434,v1:172.21.15.122:6825/3566117434] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4390009e50 con 0x7f4378073020 2024-08-19T13:19:10.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.122+0000 7f4386ffd640 1 -- 172.21.15.103:0/2763009988 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4390091050 con 0x7f4394071080 2024-08-19T13:19:10.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.122+0000 7f439a84a640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.122:6824/3566117434,v1:172.21.15.122:6825/3566117434] conn(0x7f4378073020 0x7f4378075440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.122+0000 7f4399848640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f437806be60 0x7f437806e320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4388034000 tx=0x7f438802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:10.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.123+0000 7f439a84a640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.122:6824/3566117434,v1:172.21.15.122:6825/3566117434] conn(0x7f4378073020 0x7f4378075440 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:10.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.123+0000 7f4386ffd640 1 -- 172.21.15.103:0/2763009988 <== osd.7 v2:172.21.15.122:6824/3566117434 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4390009e50 con 0x7f4378073020 2024-08-19T13:19:10.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.144+0000 7f439c2ad640 1 -- 172.21.15.103:0/2763009988 --> [v2:172.21.15.122:6824/3566117434,v1:172.21.15.122:6825/3566117434] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f43941385d0 con 0x7f4378073020 2024-08-19T13:19:10.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.145+0000 7f4386ffd640 1 -- 172.21.15.103:0/2763009988 <== osd.7 v2:172.21.15.122:6824/3566117434 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f43941385d0 con 0x7f4378073020 2024-08-19T13:19:10.146 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T13:19:10.146 INFO:teuthology.orchestra.run.smithi103.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-19T13:19:10.146 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T13:19:10.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.145+0000 7f439c2ad640 1 -- 172.21.15.103:0/2763009988 >> [v2:172.21.15.122:6824/3566117434,v1:172.21.15.122:6825/3566117434] conn(0x7f4378073020 msgr2=0x7f4378075440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.145+0000 7f439c2ad640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.122:6824/3566117434,v1:172.21.15.122:6825/3566117434] conn(0x7f4378073020 0x7f4378075440 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.146+0000 7f439c2ad640 1 -- 172.21.15.103:0/2763009988 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f437806be60 msgr2=0x7f437806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.146+0000 7f439c2ad640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f437806be60 0x7f437806e320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4388034000 tx=0x7f438802f000 comp rx=0 tx=0).stop 2024-08-19T13:19:10.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.146+0000 7f439c2ad640 1 -- 172.21.15.103:0/2763009988 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4394071080 msgr2=0x7f439413e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.146+0000 7f439c2ad640 1 --2- 172.21.15.103:0/2763009988 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4394071080 0x7f439413e530 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f439000c3f0 tx=0x7f439000c8b0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.146+0000 7f439a84a640 1 -- 172.21.15.103:0/2763009988 reap_dead start 2024-08-19T13:19:10.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.146+0000 7f439c2ad640 1 -- 172.21.15.103:0/2763009988 shutdown_connections 2024-08-19T13:19:10.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.146+0000 7f439c2ad640 1 -- 172.21.15.103:0/2763009988 >> 172.21.15.103:0/2763009988 conn(0x7f43940b1f70 msgr2=0x7f439406ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:10.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.146+0000 7f439c2ad640 1 -- 172.21.15.103:0/2763009988 shutdown_connections 2024-08-19T13:19:10.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.146+0000 7f439c2ad640 1 -- 172.21.15.103:0/2763009988 wait complete. 2024-08-19T13:19:10.160 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-08-19T13:19:10.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.285+0000 7f55ecd3d640 1 Processor -- start 2024-08-19T13:19:10.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.286+0000 7f55ecd3d640 1 -- start start 2024-08-19T13:19:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.286+0000 7f55ecd3d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f55e8137490 0x7f55e8141730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.286+0000 7f55ecd3d640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f55e8137870 0x7f55e80b1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.286+0000 7f55ecd3d640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e80a9840 con 0x7f55e80b1660 2024-08-19T13:19:10.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.286+0000 7f55ecd3d640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f55e80b28a0 con 0x7f55e8137870 2024-08-19T13:19:10.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.286+0000 7f55ecd3d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f55e8072bd0 con 0x7f55e8137490 2024-08-19T13:19:10.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.287+0000 7f55e5d9b640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f55e8137870 0x7f55e80b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.287+0000 7f55e5d9b640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f55e8137870 0x7f55e80b1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52810/0 (socket says 172.21.15.103:52810) 2024-08-19T13:19:10.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.287+0000 7f55e5d9b640 1 -- 172.21.15.103:0/2990623635 learned_addr learned my addr 172.21.15.103:0/2990623635 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:10.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.287+0000 7f55e659c640 1 --2- 172.21.15.103:0/2990623635 >> v2:172.21.15.122:3300/0 conn(0x7f55e8137490 0x7f55e8141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.287+0000 7f55e5d9b640 1 -- 172.21.15.103:0/2990623635 >> v2:172.21.15.103:3300/0 conn(0x7f55e8137870 msgr2=0x7f55e80b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T13:19:10.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.287+0000 7f55e5d9b640 1 -- 172.21.15.103:0/2990623635 >> v2:172.21.15.103:3300/0 conn(0x7f55e8137870 msgr2=0x7f55e80b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T13:19:10.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.287+0000 7f55e5d9b640 1 --2- 172.21.15.103:0/2990623635 >> v2:172.21.15.103:3300/0 conn(0x7f55e8137870 0x7f55e80b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T13:19:10.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.287+0000 7f55e5d9b640 1 --2- 172.21.15.103:0/2990623635 >> v2:172.21.15.103:3300/0 conn(0x7f55e8137870 0x7f55e80b1120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T13:19:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.287+0000 7f55e659c640 1 -- 172.21.15.103:0/2990623635 >> v2:172.21.15.103:3300/0 conn(0x7f55e8137870 msgr2=0x7f55e80b1120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T13:19:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.287+0000 7f55e659c640 1 --2- 172.21.15.103:0/2990623635 >> v2:172.21.15.103:3300/0 conn(0x7f55e8137870 0x7f55e80b1120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.287+0000 7f55e659c640 1 -- 172.21.15.103:0/2990623635 >> v1:172.21.15.103:6789/0 conn(0x7f55e80b1660 legacy=0x7f55e8071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.287+0000 7f55e659c640 1 -- 172.21.15.103:0/2990623635 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e8141210 con 0x7f55e8137490 2024-08-19T13:19:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.288+0000 7f55e659c640 1 --2- 172.21.15.103:0/2990623635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55e8137490 0x7f55e8141730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f55e8071b70 tx=0x7f55d802a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62df950a7de72233 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.288+0000 7f55e559a640 1 -- 172.21.15.103:0/2990623635 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8037070 con 0x7f55e8137490 2024-08-19T13:19:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.288+0000 7f55e559a640 1 -- 172.21.15.103:0/2990623635 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d802add0 con 0x7f55e8137490 2024-08-19T13:19:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.288+0000 7f55e559a640 1 -- 172.21.15.103:0/2990623635 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8032660 con 0x7f55e8137490 2024-08-19T13:19:10.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.289+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2990623635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55e8137490 msgr2=0x7f55e8141730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.289+0000 7f55ecd3d640 1 --2- 172.21.15.103:0/2990623635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55e8137490 0x7f55e8141730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f55e8071b70 tx=0x7f55d802a080 comp rx=0 tx=0).stop 2024-08-19T13:19:10.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.289+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2990623635 shutdown_connections 2024-08-19T13:19:10.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.289+0000 7f55ecd3d640 1 --2- 172.21.15.103:0/2990623635 >> v2:172.21.15.103:3300/0 conn(0x7f55e8137870 0x7f55e80b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.289+0000 7f55ecd3d640 1 --2- 172.21.15.103:0/2990623635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55e8137490 0x7f55e8141730 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.289+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2990623635 >> 172.21.15.103:0/2990623635 conn(0x7f55e80b6b10 msgr2=0x7f55e80b6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:10.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.289+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2990623635 shutdown_connections 2024-08-19T13:19:10.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.289+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2990623635 wait complete. 2024-08-19T13:19:10.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.290+0000 7f55ecd3d640 1 Processor -- start 2024-08-19T13:19:10.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.290+0000 7f55ecd3d640 1 -- start start 2024-08-19T13:19:10.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55ecd3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e80b1660 0x7f55e813d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55ecd3d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55e8137490 0x7f55e81398f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55ecd3d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e8137870 0x7f55e8139e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55ecd3d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55e8072470 con 0x7f55e80b1660 2024-08-19T13:19:10.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55ecd3d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55e80722f0 con 0x7f55e8137490 2024-08-19T13:19:10.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55ecd3d640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f55e80725f0 con 0x7f55e8137870 2024-08-19T13:19:10.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e80b1660 0x7f55e813d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55e5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55e8137490 0x7f55e81398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55e6d9d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e8137870 0x7f55e8139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e80b1660 0x7f55e813d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52826/0 (socket says 172.21.15.103:52826) 2024-08-19T13:19:10.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55e5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55e8137490 0x7f55e81398f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.103:40496/0 (socket says 172.21.15.103:40496) 2024-08-19T13:19:10.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55e659c640 1 -- 172.21.15.103:0/2347103062 learned_addr learned my addr 172.21.15.103:0/2347103062 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:10.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55e5d9b640 1 -- 172.21.15.103:0/2347103062 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e8137870 msgr2=0x7f55e8139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55e5d9b640 1 --2- 172.21.15.103:0/2347103062 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e8137870 0x7f55e8139e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55e5d9b640 1 -- 172.21.15.103:0/2347103062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e80b1660 msgr2=0x7f55e813d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55e5d9b640 1 --2- 172.21.15.103:0/2347103062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e80b1660 0x7f55e813d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55e5d9b640 1 -- 172.21.15.103:0/2347103062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e80aa600 con 0x7f55e8137490 2024-08-19T13:19:10.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55e659c640 1 --2- 172.21.15.103:0/2347103062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e80b1660 0x7f55e813d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:10.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.291+0000 7f55e5d9b640 1 --2- 172.21.15.103:0/2347103062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55e8137490 0x7f55e81398f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009100 tx=0x7f55dc00ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:10.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.292+0000 7f55d6ffd640 1 -- 172.21.15.103:0/2347103062 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc017070 con 0x7f55e8137490 2024-08-19T13:19:10.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.292+0000 7f55d6ffd640 1 -- 172.21.15.103:0/2347103062 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55dc004020 con 0x7f55e8137490 2024-08-19T13:19:10.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.292+0000 7f55d6ffd640 1 -- 172.21.15.103:0/2347103062 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc00f040 con 0x7f55e8137490 2024-08-19T13:19:10.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.292+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2347103062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e813a4c0 con 0x7f55e8137490 2024-08-19T13:19:10.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.292+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2347103062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e8134d80 con 0x7f55e8137490 2024-08-19T13:19:10.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.293+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2347103062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e8073e90 con 0x7f55e8137490 2024-08-19T13:19:10.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.293+0000 7f55d6ffd640 1 -- 172.21.15.103:0/2347103062 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f55dc00c6d0 con 0x7f55e8137490 2024-08-19T13:19:10.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.294+0000 7f55d6ffd640 1 --2- 172.21.15.103:0/2347103062 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f55c806be30 0x7f55c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.294+0000 7f55d6ffd640 1 -- 172.21.15.103:0/2347103062 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7f55dc08a880 con 0x7f55e8137490 2024-08-19T13:19:10.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.294+0000 7f55e659c640 1 --2- 172.21.15.103:0/2347103062 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f55c806be30 0x7f55c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.294+0000 7f55e659c640 1 --2- 172.21.15.103:0/2347103062 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f55c806be30 0x7f55c806e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009560 tx=0x7f55d803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:10.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.298+0000 7f55d6ffd640 1 -- 172.21.15.103:0/2347103062 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f55e8073e90 con 0x7f55e8137490 2024-08-19T13:19:10.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.466+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2347103062 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f55e8141730 con 0x7f55c806be30 2024-08-19T13:19:10.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.467+0000 7f55d6ffd640 1 -- 172.21.15.103:0/2347103062 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f55e8141730 con 0x7f55c806be30 2024-08-19T13:19:10.467 INFO:teuthology.orchestra.run.smithi103.stderr:instructed osd(s) 7 to deep-scrub 2024-08-19T13:19:10.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.468+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2347103062 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f55c806be30 msgr2=0x7f55c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.468+0000 7f55ecd3d640 1 --2- 172.21.15.103:0/2347103062 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f55c806be30 0x7f55c806e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009560 tx=0x7f55d803c040 comp rx=0 tx=0).stop 2024-08-19T13:19:10.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.468+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2347103062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55e8137490 msgr2=0x7f55e81398f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.468+0000 7f55ecd3d640 1 --2- 172.21.15.103:0/2347103062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55e8137490 0x7f55e81398f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009100 tx=0x7f55dc00ec70 comp rx=0 tx=0).stop 2024-08-19T13:19:10.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.468+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2347103062 shutdown_connections 2024-08-19T13:19:10.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.468+0000 7f55ecd3d640 1 --2- 172.21.15.103:0/2347103062 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f55c806be30 0x7f55c806e2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.468+0000 7f55ecd3d640 1 --2- 172.21.15.103:0/2347103062 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e8137870 0x7f55e8139e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.468+0000 7f55ecd3d640 1 --2- 172.21.15.103:0/2347103062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55e8137490 0x7f55e81398f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.468+0000 7f55ecd3d640 1 --2- 172.21.15.103:0/2347103062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e80b1660 0x7f55e813d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.468+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2347103062 >> 172.21.15.103:0/2347103062 conn(0x7f55e80b6b10 msgr2=0x7f55e80b6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:10.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.469+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2347103062 shutdown_connections 2024-08-19T13:19:10.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.469+0000 7f55ecd3d640 1 -- 172.21.15.103:0/2347103062 wait complete. 2024-08-19T13:19:10.480 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T13:19:10.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.579+0000 7fe7f52fe640 1 Processor -- start 2024-08-19T13:19:10.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.579+0000 7fe7f52fe640 1 -- start start 2024-08-19T13:19:10.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.580+0000 7fe7f52fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7f00b5c10 0x7fe7f0070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.580+0000 7fe7f52fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe7f00b41a0 0x7fe7f0070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.580+0000 7fe7f52fe640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f00b5ff0 con 0x7fe7f0141fb0 2024-08-19T13:19:10.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.580+0000 7fe7f52fe640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f0072410 con 0x7fe7f00b41a0 2024-08-19T13:19:10.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.580+0000 7fe7f52fe640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f00b2d40 con 0x7fe7f00b5c10 2024-08-19T13:19:10.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.580+0000 7fe7eeffd640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7f00b5c10 0x7fe7f0070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.580+0000 7fe7ef7fe640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fe7f0141fb0 0x7fe7f0142460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:43000/0 (socket says 172.21.15.103:43000) 2024-08-19T13:19:10.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.580+0000 7fe7ef7fe640 1 -- 172.21.15.103:0/138828072 learned_addr learned my addr 172.21.15.103:0/138828072 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-08-19T13:19:10.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.580+0000 7fe7ee7fc640 1 --2- 172.21.15.103:0/138828072 >> v2:172.21.15.122:3300/0 conn(0x7fe7f00b41a0 0x7fe7f0070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.580+0000 7fe7eeffd640 1 -- 172.21.15.103:0/138828072 >> v2:172.21.15.122:3300/0 conn(0x7fe7f00b41a0 msgr2=0x7fe7f0070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:10.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.580+0000 7fe7edffb640 1 -- 172.21.15.103:0/138828072 <== mon.0 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3194281668 0 0) 0x7fe7f00b5ff0 con 0x7fe7f0141fb0 2024-08-19T13:19:10.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.580+0000 7fe7eeffd640 1 --2- 172.21.15.103:0/138828072 >> v2:172.21.15.122:3300/0 conn(0x7fe7f00b41a0 0x7fe7f0070bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.580+0000 7fe7eeffd640 1 -- 172.21.15.103:0/138828072 >> v1:172.21.15.103:6789/0 conn(0x7fe7f0141fb0 legacy=0x7fe7f0142460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.580+0000 7fe7eeffd640 1 -- 172.21.15.103:0/138828072 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f0073af0 con 0x7fe7f00b5c10 2024-08-19T13:19:10.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.581+0000 7fe7eeffd640 1 --2- 172.21.15.103:0/138828072 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7f00b5c10 0x7fe7f0070680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00097a0 tx=0x7fe7e002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=646a6a8e7b4b67f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:10.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.581+0000 7fe7edffb640 1 -- 172.21.15.103:0/138828072 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0037070 con 0x7fe7f00b5c10 2024-08-19T13:19:10.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.581+0000 7fe7edffb640 1 -- 172.21.15.103:0/138828072 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e002ae50 con 0x7fe7f00b5c10 2024-08-19T13:19:10.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.581+0000 7fe7edffb640 1 -- 172.21.15.103:0/138828072 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0032660 con 0x7fe7f00b5c10 2024-08-19T13:19:10.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.581+0000 7fe7f52fe640 1 -- 172.21.15.103:0/138828072 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7f00b5c10 msgr2=0x7fe7f0070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.581+0000 7fe7f52fe640 1 --2- 172.21.15.103:0/138828072 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7f00b5c10 0x7fe7f0070680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00097a0 tx=0x7fe7e002a100 comp rx=0 tx=0).stop 2024-08-19T13:19:10.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.581+0000 7fe7f52fe640 1 -- 172.21.15.103:0/138828072 shutdown_connections 2024-08-19T13:19:10.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.581+0000 7fe7f52fe640 1 --2- 172.21.15.103:0/138828072 >> v2:172.21.15.122:3300/0 conn(0x7fe7f00b41a0 0x7fe7f0070bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.581+0000 7fe7f52fe640 1 --2- 172.21.15.103:0/138828072 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7f00b5c10 0x7fe7f0070680 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.581+0000 7fe7f52fe640 1 -- 172.21.15.103:0/138828072 >> 172.21.15.103:0/138828072 conn(0x7fe7f00b8dc0 msgr2=0x7fe7f00b5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:10.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.582+0000 7fe7f52fe640 1 -- 172.21.15.103:0/138828072 shutdown_connections 2024-08-19T13:19:10.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.582+0000 7fe7f52fe640 1 -- 172.21.15.103:0/138828072 wait complete. 2024-08-19T13:19:10.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.583+0000 7fe7f52fe640 1 Processor -- start 2024-08-19T13:19:10.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.583+0000 7fe7f52fe640 1 -- start start 2024-08-19T13:19:10.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.583+0000 7fe7f52fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f00b41a0 0x7fe7f0146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.583+0000 7fe7f52fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7f00b5c10 0x7fe7f01440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.583+0000 7fe7f52fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7f0141fb0 0x7fe7f0144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.583+0000 7fe7f52fe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f006fb10 con 0x7fe7f0141fb0 2024-08-19T13:19:10.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.583+0000 7fe7f52fe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f006fc90 con 0x7fe7f00b41a0 2024-08-19T13:19:10.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.583+0000 7fe7f52fe640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f014d550 con 0x7fe7f00b5c10 2024-08-19T13:19:10.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.583+0000 7fe7ee7fc640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7f00b5c10 0x7fe7f01440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.583+0000 7fe7ef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7f0141fb0 0x7fe7f0144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.583+0000 7fe7ee7fc640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7f00b5c10 0x7fe7f01440c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41728/0 (socket says 172.21.15.103:41728) 2024-08-19T13:19:10.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.583+0000 7fe7ee7fc640 1 -- 172.21.15.103:0/537742968 learned_addr learned my addr 172.21.15.103:0/537742968 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:10.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.583+0000 7fe7eeffd640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f00b41a0 0x7fe7f0146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.584+0000 7fe7ee7fc640 1 -- 172.21.15.103:0/537742968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f00b41a0 msgr2=0x7fe7f0146a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.584+0000 7fe7ee7fc640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f00b41a0 0x7fe7f0146a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.584+0000 7fe7ee7fc640 1 -- 172.21.15.103:0/537742968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7f0141fb0 msgr2=0x7fe7f0144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.584+0000 7fe7ee7fc640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7f0141fb0 0x7fe7f0144600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.584+0000 7fe7ee7fc640 1 -- 172.21.15.103:0/537742968 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f013a6c0 con 0x7fe7f00b5c10 2024-08-19T13:19:10.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.584+0000 7fe7eeffd640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f00b41a0 0x7fe7f0146a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T13:19:10.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.584+0000 7fe7ef7fe640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7f0141fb0 0x7fe7f0144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:10.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.584+0000 7fe7ee7fc640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7f00b5c10 0x7fe7f01440c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400e980 tx=0x7fe7e400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:10.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.584+0000 7fe7d77fe640 1 -- 172.21.15.103:0/537742968 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4017070 con 0x7fe7f00b5c10 2024-08-19T13:19:10.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.584+0000 7fe7d77fe640 1 -- 172.21.15.103:0/537742968 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e400ca00 con 0x7fe7f00b5c10 2024-08-19T13:19:10.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.584+0000 7fe7f52fe640 1 -- 172.21.15.103:0/537742968 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f01389b0 con 0x7fe7f00b5c10 2024-08-19T13:19:10.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.584+0000 7fe7d77fe640 1 -- 172.21.15.103:0/537742968 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e400cdd0 con 0x7fe7f00b5c10 2024-08-19T13:19:10.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.584+0000 7fe7f52fe640 1 -- 172.21.15.103:0/537742968 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f013e180 con 0x7fe7f00b5c10 2024-08-19T13:19:10.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.585+0000 7fe7f52fe640 1 -- 172.21.15.103:0/537742968 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f0050530 con 0x7fe7f00b5c10 2024-08-19T13:19:10.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.585+0000 7fe7d77fe640 1 -- 172.21.15.103:0/537742968 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe7e401a070 con 0x7fe7f00b5c10 2024-08-19T13:19:10.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.586+0000 7fe7d77fe640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe7dc06be60 0x7fe7dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:10.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.586+0000 7fe7d77fe640 1 -- 172.21.15.103:0/537742968 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7fe7e408ada0 con 0x7fe7f00b5c10 2024-08-19T13:19:10.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.586+0000 7fe7eeffd640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe7dc06be60 0x7fe7dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:10.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.586+0000 7fe7eeffd640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe7dc06be60 0x7fe7dc06e320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0034000 tx=0x7fe7e002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:10.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.590+0000 7fe7d77fe640 1 -- 172.21.15.103:0/537742968 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7fe7f0050530 con 0x7fe7f00b5c10 2024-08-19T13:19:10.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.756+0000 7fe7f52fe640 1 -- 172.21.15.103:0/537742968 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7f0075a00 con 0x7fe7dc06be60 2024-08-19T13:19:10.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.759+0000 7fe7d77fe640 1 -- 172.21.15.103:0/537742968 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117271 (secure 0 0 0) 0x7fe7e005ea30 con 0x7fe7dc06be60 2024-08-19T13:19:10.761 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:19:10.761 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-08-19T13:19:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.761+0000 7fe7f52fe640 1 -- 172.21.15.103:0/537742968 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe7dc06be60 msgr2=0x7fe7dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.761+0000 7fe7f52fe640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe7dc06be60 0x7fe7dc06e320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0034000 tx=0x7fe7e002e000 comp rx=0 tx=0).stop 2024-08-19T13:19:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.761+0000 7fe7f52fe640 1 -- 172.21.15.103:0/537742968 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7f00b5c10 msgr2=0x7fe7f01440c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.761+0000 7fe7f52fe640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7f00b5c10 0x7fe7f01440c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400e980 tx=0x7fe7e400ee40 comp rx=0 tx=0).stop 2024-08-19T13:19:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.761+0000 7fe7f52fe640 1 -- 172.21.15.103:0/537742968 shutdown_connections 2024-08-19T13:19:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.761+0000 7fe7f52fe640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7fe7dc06be60 0x7fe7dc06e320 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.761+0000 7fe7f52fe640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7f0141fb0 0x7fe7f0144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.761+0000 7fe7f52fe640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7f00b5c10 0x7fe7f01440c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.761+0000 7fe7f52fe640 1 --2- 172.21.15.103:0/537742968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f00b41a0 0x7fe7f0146a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.761+0000 7fe7f52fe640 1 -- 172.21.15.103:0/537742968 >> 172.21.15.103:0/537742968 conn(0x7fe7f00b8dc0 msgr2=0x7fe7f0143090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.761+0000 7fe7f52fe640 1 -- 172.21.15.103:0/537742968 shutdown_connections 2024-08-19T13:19:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:10.761+0000 7fe7f52fe640 1 -- 172.21.15.103:0/537742968 wait complete. 2024-08-19T13:19:10.773 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-08-19T13:19:10.048544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3575952171,"num_objects":883,"num_object_clones":0,"num_object_copies":1766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":883,"num_whiteouts":0,"num_read":2689,"num_read_kb":2689,"num_write":162641,"num_write_kb":5650019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":165536,"ondisk_log_size":165536,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9310068,"kb_used_data":6676064,"kb_used_omap":12,"kb_used_meta":2633907,"kb_avail":745664652,"statfs":{"total":773094113280,"available":763560603648,"internally_reserved":0,"allocated":6836289536,"data_stored":6851322542,"data_compressed":70441838,"data_compressed_allocated":73416704,"data_compressed_original":94011392,"omap_allocated":12752,"internal_metadata":2697121328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8720,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":66,"num_read_kb":71,"num_write":17,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002049"},"pg_stats":[{"pgid":"3.1f","version":"22'3551","reported_seq":3580,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.907684+0000","last_change":"2024-08-19T13:15:04.704168+0000","last_active":"2024-08-19T13:19:03.907684+0000","last_peered":"2024-08-19T13:19:03.907684+0000","last_clean":"2024-08-19T13:19:03.907684+0000","last_became_active":"2024-08-19T13:15:04.703897+0000","last_became_peered":"2024-08-19T13:15:04.703897+0000","last_unstale":"2024-08-19T13:19:03.907684+0000","last_undegraded":"2024-08-19T13:19:03.907684+0000","last_fullsized":"2024-08-19T13:19:03.907684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3551,"log_dups_size":0,"ondisk_log_size":3551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":3566,"num_write_kb":193154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"24'5749","reported_seq":5771,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:04.218957+0000","last_change":"2024-08-19T13:15:04.702034+0000","last_active":"2024-08-19T13:19:04.218957+0000","last_peered":"2024-08-19T13:19:04.218957+0000","last_clean":"2024-08-19T13:19:04.218957+0000","last_became_active":"2024-08-19T13:15:04.701653+0000","last_became_peered":"2024-08-19T13:15:04.701653+0000","last_unstale":"2024-08-19T13:19:04.218957+0000","last_undegraded":"2024-08-19T13:19:04.218957+0000","last_fullsized":"2024-08-19T13:19:04.218957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5749,"log_dups_size":0,"ondisk_log_size":5749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128583504,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5733,"num_write_kb":196004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"24'5875","reported_seq":5893,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.910880+0000","last_change":"2024-08-19T13:15:04.708047+0000","last_active":"2024-08-19T13:19:03.910880+0000","last_peered":"2024-08-19T13:19:03.910880+0000","last_clean":"2024-08-19T13:19:03.910880+0000","last_became_active":"2024-08-19T13:15:04.707756+0000","last_became_peered":"2024-08-19T13:15:04.707756+0000","last_unstale":"2024-08-19T13:19:03.910880+0000","last_undegraded":"2024-08-19T13:19:03.910880+0000","last_fullsized":"2024-08-19T13:19:03.910880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5875,"log_dups_size":0,"ondisk_log_size":5875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:23:19.211747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113312264,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5859,"num_write_kb":177354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"22'1998","reported_seq":2015,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.909695+0000","last_change":"2024-08-19T13:15:04.707039+0000","last_active":"2024-08-19T13:19:03.909695+0000","last_peered":"2024-08-19T13:19:03.909695+0000","last_clean":"2024-08-19T13:19:03.909695+0000","last_became_active":"2024-08-19T13:15:04.706726+0000","last_became_peered":"2024-08-19T13:15:04.706726+0000","last_unstale":"2024-08-19T13:19:03.909695+0000","last_undegraded":"2024-08-19T13:19:03.909695+0000","last_fullsized":"2024-08-19T13:19:03.909695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":1998,"log_dups_size":0,"ondisk_log_size":1998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:45:09.075439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1990,"num_write_kb":94721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"24'3783","reported_seq":3802,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.914845+0000","last_change":"2024-08-19T13:15:04.707093+0000","last_active":"2024-08-19T13:19:03.914845+0000","last_peered":"2024-08-19T13:19:03.914845+0000","last_clean":"2024-08-19T13:19:03.914845+0000","last_became_active":"2024-08-19T13:15:04.706832+0000","last_became_peered":"2024-08-19T13:15:04.706832+0000","last_unstale":"2024-08-19T13:19:03.914845+0000","last_undegraded":"2024-08-19T13:19:03.914845+0000","last_fullsized":"2024-08-19T13:19:03.914845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3783,"log_dups_size":0,"ondisk_log_size":3783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104858112,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3741,"num_write_kb":132744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"24'7048","reported_seq":7070,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.908376+0000","last_change":"2024-08-19T13:15:04.706943+0000","last_active":"2024-08-19T13:19:03.908376+0000","last_peered":"2024-08-19T13:19:03.908376+0000","last_clean":"2024-08-19T13:19:03.908376+0000","last_became_active":"2024-08-19T13:15:04.706371+0000","last_became_peered":"2024-08-19T13:15:04.706371+0000","last_unstale":"2024-08-19T13:19:03.908376+0000","last_undegraded":"2024-08-19T13:19:03.908376+0000","last_fullsized":"2024-08-19T13:19:03.908376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":7048,"log_dups_size":0,"ondisk_log_size":7048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134217728,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":7029,"num_write_kb":256635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"24'4611","reported_seq":4631,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.910893+0000","last_change":"2024-08-19T13:15:04.701266+0000","last_active":"2024-08-19T13:19:03.910893+0000","last_peered":"2024-08-19T13:19:03.910893+0000","last_clean":"2024-08-19T13:19:03.910893+0000","last_became_active":"2024-08-19T13:15:04.700993+0000","last_became_peered":"2024-08-19T13:15:04.700993+0000","last_unstale":"2024-08-19T13:19:03.910893+0000","last_undegraded":"2024-08-19T13:19:03.910893+0000","last_fullsized":"2024-08-19T13:19:03.910893+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4611,"log_dups_size":0,"ondisk_log_size":4611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109052416,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":4571,"num_write_kb":180994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"23'6372","reported_seq":6390,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.860285+0000","last_change":"2024-08-19T13:15:04.699913+0000","last_active":"2024-08-19T13:19:03.860285+0000","last_peered":"2024-08-19T13:19:03.860285+0000","last_clean":"2024-08-19T13:19:03.860285+0000","last_became_active":"2024-08-19T13:15:04.699219+0000","last_became_peered":"2024-08-19T13:15:04.699219+0000","last_unstale":"2024-08-19T13:19:03.860285+0000","last_undegraded":"2024-08-19T13:19:03.860285+0000","last_fullsized":"2024-08-19T13:19:03.860285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":6372,"log_dups_size":0,"ondisk_log_size":6372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6366,"num_write_kb":136778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"24'7679","reported_seq":7705,"reported_epoch":24,"state":"active+clean+scrubbing+deep","last_fresh":"2024-08-19T13:19:09.260499+0000","last_change":"2024-08-19T13:19:09.003341+0000","last_active":"2024-08-19T13:19:09.260499+0000","last_peered":"2024-08-19T13:19:09.260499+0000","last_clean":"2024-08-19T13:19:09.260499+0000","last_became_active":"2024-08-19T13:15:04.700904+0000","last_became_peered":"2024-08-19T13:15:04.700904+0000","last_unstale":"2024-08-19T13:19:09.260499+0000","last_undegraded":"2024-08-19T13:19:09.260499+0000","last_fullsized":"2024-08-19T13:19:09.260499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":24,"log_size":7679,"log_dups_size":0,"ondisk_log_size":7679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146932240,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":7658,"num_write_kb":230670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"24'3890","reported_seq":3907,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.910273+0000","last_change":"2024-08-19T13:15:04.704857+0000","last_active":"2024-08-19T13:19:03.910273+0000","last_peered":"2024-08-19T13:19:03.910273+0000","last_clean":"2024-08-19T13:19:03.910273+0000","last_became_active":"2024-08-19T13:15:04.704383+0000","last_became_peered":"2024-08-19T13:15:04.704383+0000","last_unstale":"2024-08-19T13:19:03.910273+0000","last_undegraded":"2024-08-19T13:19:03.910273+0000","last_fullsized":"2024-08-19T13:19:03.910273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3890,"log_dups_size":0,"ondisk_log_size":3890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:27:48.271684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106366632,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3864,"num_write_kb":162549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"23'8294","reported_seq":8334,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.874947+0000","last_change":"2024-08-19T13:15:04.703573+0000","last_active":"2024-08-19T13:19:03.874947+0000","last_peered":"2024-08-19T13:19:03.874947+0000","last_clean":"2024-08-19T13:19:03.874947+0000","last_became_active":"2024-08-19T13:15:04.703277+0000","last_became_peered":"2024-08-19T13:15:04.703277+0000","last_unstale":"2024-08-19T13:19:03.874947+0000","last_undegraded":"2024-08-19T13:19:03.874947+0000","last_fullsized":"2024-08-19T13:19:03.874947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":8294,"log_dups_size":0,"ondisk_log_size":8294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:08:33.844086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125960192,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":85,"num_read_kb":132,"num_write":8326,"num_write_kb":182642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"24'6459","reported_seq":6578,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.914597+0000","last_change":"2024-08-19T13:15:04.707221+0000","last_active":"2024-08-19T13:19:03.914597+0000","last_peered":"2024-08-19T13:19:03.914597+0000","last_clean":"2024-08-19T13:19:03.914597+0000","last_became_active":"2024-08-19T13:15:04.706496+0000","last_became_peered":"2024-08-19T13:15:04.706496+0000","last_unstale":"2024-08-19T13:19:03.914597+0000","last_undegraded":"2024-08-19T13:19:03.914597+0000","last_fullsized":"2024-08-19T13:19:03.914597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":6459,"log_dups_size":0,"ondisk_log_size":6459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90964496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":6419,"num_write_kb":175583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.905528+0000","last_change":"2024-08-19T13:14:58.168503+0000","last_active":"2024-08-19T13:19:03.905528+0000","last_peered":"2024-08-19T13:19:03.905528+0000","last_clean":"2024-08-19T13:19:03.905528+0000","last_became_active":"2024-08-19T13:14:56.044662+0000","last_became_peered":"2024-08-19T13:14:56.044662+0000","last_unstale":"2024-08-19T13:19:03.905528+0000","last_undegraded":"2024-08-19T13:19:03.905528+0000","last_fullsized":"2024-08-19T13:19:03.905528+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T23:26:23.570412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039371300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'6818","reported_seq":6854,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:08.246875+0000","last_change":"2024-08-19T13:19:08.246904+0000","last_active":"2024-08-19T13:19:08.246875+0000","last_peered":"2024-08-19T13:19:08.246875+0000","last_clean":"2024-08-19T13:19:08.246875+0000","last_became_active":"2024-08-19T13:15:04.700540+0000","last_became_peered":"2024-08-19T13:15:04.700540+0000","last_unstale":"2024-08-19T13:19:08.246875+0000","last_undegraded":"2024-08-19T13:19:08.246875+0000","last_fullsized":"2024-08-19T13:19:08.246875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'6818","last_scrub_stamp":"2024-08-19T13:19:08.246803+0000","last_deep_scrub":"24'6818","last_deep_scrub_stamp":"2024-08-19T13:19:08.246803+0000","last_clean_scrub_stamp":"2024-08-19T13:19:08.246803+0000","objects_scrubbed":26,"log_size":6818,"log_dups_size":0,"ondisk_log_size":6818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:16:21.566219+0000","scrub_duration":277,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104988672,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":34,"num_read_kb":151,"num_write":6787,"num_write_kb":141259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:07.841151+0000","last_change":"2024-08-19T13:19:07.841169+0000","last_active":"2024-08-19T13:19:07.841151+0000","last_peered":"2024-08-19T13:19:07.841151+0000","last_clean":"2024-08-19T13:19:07.841151+0000","last_became_active":"2024-08-19T13:14:56.045731+0000","last_became_peered":"2024-08-19T13:14:56.045731+0000","last_unstale":"2024-08-19T13:19:07.841151+0000","last_undegraded":"2024-08-19T13:19:07.841151+0000","last_fullsized":"2024-08-19T13:19:07.841151+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:19:07.841083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:19:07.841083+0000","last_clean_scrub_stamp":"2024-08-19T13:19:07.841083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T15:45:07.439179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044000000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"24'5381","reported_seq":5407,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.912307+0000","last_change":"2024-08-19T13:15:04.703333+0000","last_active":"2024-08-19T13:19:03.912307+0000","last_peered":"2024-08-19T13:19:03.912307+0000","last_clean":"2024-08-19T13:19:03.912307+0000","last_became_active":"2024-08-19T13:15:04.703011+0000","last_became_peered":"2024-08-19T13:15:04.703011+0000","last_unstale":"2024-08-19T13:19:03.912307+0000","last_undegraded":"2024-08-19T13:19:03.912307+0000","last_fullsized":"2024-08-19T13:19:03.912307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5381,"log_dups_size":0,"ondisk_log_size":5381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146276352,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5343,"num_write_kb":213509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.875439+0000","last_change":"2024-08-19T13:14:58.373877+0000","last_active":"2024-08-19T13:19:03.875439+0000","last_peered":"2024-08-19T13:19:03.875439+0000","last_clean":"2024-08-19T13:19:03.875439+0000","last_became_active":"2024-08-19T13:14:56.111427+0000","last_became_peered":"2024-08-19T13:14:56.111427+0000","last_unstale":"2024-08-19T13:19:03.875439+0000","last_undegraded":"2024-08-19T13:19:03.875439+0000","last_fullsized":"2024-08-19T13:19:03.875439+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:19:29.396593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050374000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'8281","reported_seq":8299,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.860696+0000","last_change":"2024-08-19T13:15:04.699221+0000","last_active":"2024-08-19T13:19:03.860696+0000","last_peered":"2024-08-19T13:19:03.860696+0000","last_clean":"2024-08-19T13:19:03.860696+0000","last_became_active":"2024-08-19T13:15:04.698944+0000","last_became_peered":"2024-08-19T13:15:04.698944+0000","last_unstale":"2024-08-19T13:19:03.860696+0000","last_undegraded":"2024-08-19T13:19:03.860696+0000","last_fullsized":"2024-08-19T13:19:03.860696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":8281,"log_dups_size":0,"ondisk_log_size":8281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8278,"num_write_kb":279553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:08.848399+0000","last_change":"2024-08-19T13:19:08.848399+0000","last_active":"2024-08-19T13:19:08.848399+0000","last_peered":"2024-08-19T13:19:08.848399+0000","last_clean":"2024-08-19T13:19:08.848399+0000","last_became_active":"2024-08-19T13:14:56.111065+0000","last_became_peered":"2024-08-19T13:14:56.111065+0000","last_unstale":"2024-08-19T13:19:08.848399+0000","last_undegraded":"2024-08-19T13:19:08.848399+0000","last_fullsized":"2024-08-19T13:19:08.848399+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:19:08.848257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:19:08.848257+0000","last_clean_scrub_stamp":"2024-08-19T13:19:08.848257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:47:53.679166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042019699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'4283","reported_seq":4314,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.911103+0000","last_change":"2024-08-19T13:15:04.698014+0000","last_active":"2024-08-19T13:19:03.911103+0000","last_peered":"2024-08-19T13:19:03.911103+0000","last_clean":"2024-08-19T13:19:03.911103+0000","last_became_active":"2024-08-19T13:15:04.697675+0000","last_became_peered":"2024-08-19T13:15:04.697675+0000","last_unstale":"2024-08-19T13:19:03.911103+0000","last_undegraded":"2024-08-19T13:19:03.911103+0000","last_fullsized":"2024-08-19T13:19:03.911103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4283,"log_dups_size":0,"ondisk_log_size":4283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:28:29.123627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114253824,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":58,"num_read_kb":45,"num_write":4306,"num_write_kb":159578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'5377","reported_seq":5410,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.864186+0000","last_change":"2024-08-19T13:15:04.703723+0000","last_active":"2024-08-19T13:19:03.864186+0000","last_peered":"2024-08-19T13:19:03.864186+0000","last_clean":"2024-08-19T13:19:03.864186+0000","last_became_active":"2024-08-19T13:15:04.703417+0000","last_became_peered":"2024-08-19T13:15:04.703417+0000","last_unstale":"2024-08-19T13:19:03.864186+0000","last_undegraded":"2024-08-19T13:19:03.864186+0000","last_fullsized":"2024-08-19T13:19:03.864186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5377,"log_dups_size":0,"ondisk_log_size":5377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":5405,"num_write_kb":165854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":35,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.910712+0000","last_change":"2024-08-19T13:14:58.056552+0000","last_active":"2024-08-19T13:19:03.910712+0000","last_peered":"2024-08-19T13:19:03.910712+0000","last_clean":"2024-08-19T13:19:03.910712+0000","last_became_active":"2024-08-19T13:14:56.045753+0000","last_became_peered":"2024-08-19T13:14:56.045753+0000","last_unstale":"2024-08-19T13:19:03.910712+0000","last_undegraded":"2024-08-19T13:19:03.910712+0000","last_fullsized":"2024-08-19T13:19:03.910712+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:12:21.860038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037158599999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"24'2825","reported_seq":2849,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:04.126248+0000","last_change":"2024-08-19T13:15:04.699545+0000","last_active":"2024-08-19T13:19:04.126248+0000","last_peered":"2024-08-19T13:19:04.126248+0000","last_clean":"2024-08-19T13:19:04.126248+0000","last_became_active":"2024-08-19T13:15:04.699255+0000","last_became_peered":"2024-08-19T13:15:04.699255+0000","last_unstale":"2024-08-19T13:19:04.126248+0000","last_undegraded":"2024-08-19T13:19:04.126248+0000","last_fullsized":"2024-08-19T13:19:04.126248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":2825,"log_dups_size":0,"ondisk_log_size":2825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84017152,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":2784,"num_write_kb":135939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:09.024609+0000","last_change":"2024-08-19T13:19:09.024609+0000","last_active":"2024-08-19T13:19:09.024609+0000","last_peered":"2024-08-19T13:19:09.024609+0000","last_clean":"2024-08-19T13:19:09.024609+0000","last_became_active":"2024-08-19T13:14:56.046524+0000","last_became_peered":"2024-08-19T13:14:56.046524+0000","last_unstale":"2024-08-19T13:19:09.024609+0000","last_undegraded":"2024-08-19T13:19:09.024609+0000","last_fullsized":"2024-08-19T13:19:09.024609+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:19:09.024522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:19:09.024522+0000","last_clean_scrub_stamp":"2024-08-19T13:19:09.024522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:31:08.377525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043587900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"24'5931","reported_seq":5950,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.914706+0000","last_change":"2024-08-19T13:15:04.710024+0000","last_active":"2024-08-19T13:19:03.914706+0000","last_peered":"2024-08-19T13:19:03.914706+0000","last_clean":"2024-08-19T13:19:03.914706+0000","last_became_active":"2024-08-19T13:15:04.709768+0000","last_became_peered":"2024-08-19T13:15:04.709768+0000","last_unstale":"2024-08-19T13:19:03.914706+0000","last_undegraded":"2024-08-19T13:19:03.914706+0000","last_fullsized":"2024-08-19T13:19:03.914706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5931,"log_dups_size":0,"ondisk_log_size":5931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146866696,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5915,"num_write_kb":254104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.874802+0000","last_change":"2024-08-19T13:14:58.373682+0000","last_active":"2024-08-19T13:19:03.874802+0000","last_peered":"2024-08-19T13:19:03.874802+0000","last_clean":"2024-08-19T13:19:03.874802+0000","last_became_active":"2024-08-19T13:14:56.045391+0000","last_became_peered":"2024-08-19T13:14:56.045391+0000","last_unstale":"2024-08-19T13:19:03.874802+0000","last_undegraded":"2024-08-19T13:19:03.874802+0000","last_fullsized":"2024-08-19T13:19:03.874802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:41:33.266324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037776800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"24'5404","reported_seq":5444,"reported_epoch":24,"state":"active+clean+scrubbing+deep","last_fresh":"2024-08-19T13:19:09.218566+0000","last_change":"2024-08-19T13:19:08.953506+0000","last_active":"2024-08-19T13:19:09.218566+0000","last_peered":"2024-08-19T13:19:09.218566+0000","last_clean":"2024-08-19T13:19:09.218566+0000","last_became_active":"2024-08-19T13:15:04.698015+0000","last_became_peered":"2024-08-19T13:15:04.698015+0000","last_unstale":"2024-08-19T13:19:09.218566+0000","last_undegraded":"2024-08-19T13:19:09.218566+0000","last_fullsized":"2024-08-19T13:19:09.218566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":29,"log_size":5404,"log_dups_size":0,"ondisk_log_size":5404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":68,"num_read_kb":114,"num_write":5405,"num_write_kb":156434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":34,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.909612+0000","last_change":"2024-08-19T13:14:58.056578+0000","last_active":"2024-08-19T13:19:03.909612+0000","last_peered":"2024-08-19T13:19:03.909612+0000","last_clean":"2024-08-19T13:19:03.909612+0000","last_became_active":"2024-08-19T13:14:56.045464+0000","last_became_peered":"2024-08-19T13:14:56.045464+0000","last_unstale":"2024-08-19T13:19:03.909612+0000","last_undegraded":"2024-08-19T13:19:03.909612+0000","last_fullsized":"2024-08-19T13:19:03.909612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:55.033447+0000","last_clean_scrub_stamp":"2024-08-19T13:14:55.033447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:35:16.028455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039910299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.874808+0000","last_change":"2024-08-19T13:14:55.043964+0000","last_active":"2024-08-19T13:19:03.874808+0000","last_peered":"2024-08-19T13:19:03.874808+0000","last_clean":"2024-08-19T13:19:03.874808+0000","last_became_active":"2024-08-19T13:14:55.043626+0000","last_became_peered":"2024-08-19T13:14:55.043626+0000","last_unstale":"2024-08-19T13:19:03.874808+0000","last_undegraded":"2024-08-19T13:19:03.874808+0000","last_fullsized":"2024-08-19T13:19:03.874808+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:14:54.027963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:14:54.027963+0000","last_clean_scrub_stamp":"2024-08-19T13:14:54.027963+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:36:05.876096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"24'2770","reported_seq":2788,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.911820+0000","last_change":"2024-08-19T13:15:04.699203+0000","last_active":"2024-08-19T13:19:03.911820+0000","last_peered":"2024-08-19T13:19:03.911820+0000","last_clean":"2024-08-19T13:19:03.911820+0000","last_became_active":"2024-08-19T13:15:04.698903+0000","last_became_peered":"2024-08-19T13:15:04.698903+0000","last_unstale":"2024-08-19T13:19:03.911820+0000","last_undegraded":"2024-08-19T13:19:03.911820+0000","last_fullsized":"2024-08-19T13:19:03.911820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":2770,"log_dups_size":0,"ondisk_log_size":2770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2699,"num_write_kb":115587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"24'9943","reported_seq":10084,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:04.133973+0000","last_change":"2024-08-19T13:15:04.708930+0000","last_active":"2024-08-19T13:19:04.133973+0000","last_peered":"2024-08-19T13:19:04.133973+0000","last_clean":"2024-08-19T13:19:04.133973+0000","last_became_active":"2024-08-19T13:15:04.706442+0000","last_became_peered":"2024-08-19T13:15:04.706442+0000","last_unstale":"2024-08-19T13:19:04.133973+0000","last_undegraded":"2024-08-19T13:19:04.133973+0000","last_fullsized":"2024-08-19T13:19:04.133973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":9943,"log_dups_size":0,"ondisk_log_size":9943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145424384,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":386,"num_read_kb":352,"num_write":6877,"num_write_kb":212562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"24'5337","reported_seq":5357,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.905372+0000","last_change":"2024-08-19T13:15:04.708013+0000","last_active":"2024-08-19T13:19:03.905372+0000","last_peered":"2024-08-19T13:19:03.905372+0000","last_clean":"2024-08-19T13:19:03.905372+0000","last_became_active":"2024-08-19T13:15:04.707724+0000","last_became_peered":"2024-08-19T13:15:04.707724+0000","last_unstale":"2024-08-19T13:19:03.905372+0000","last_undegraded":"2024-08-19T13:19:03.905372+0000","last_fullsized":"2024-08-19T13:19:03.905372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":5337,"log_dups_size":0,"ondisk_log_size":5337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:59:28.795104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":11,"num_read_kb":71,"num_write":5419,"num_write_kb":161478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"22'3547","reported_seq":3565,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.860646+0000","last_change":"2024-08-19T13:15:04.700346+0000","last_active":"2024-08-19T13:19:03.860646+0000","last_peered":"2024-08-19T13:19:03.860646+0000","last_clean":"2024-08-19T13:19:03.860646+0000","last_became_active":"2024-08-19T13:15:04.699228+0000","last_became_peered":"2024-08-19T13:15:04.699228+0000","last_unstale":"2024-08-19T13:19:03.860646+0000","last_undegraded":"2024-08-19T13:19:03.860646+0000","last_fullsized":"2024-08-19T13:19:03.860646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3547,"log_dups_size":0,"ondisk_log_size":3547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3536,"num_write_kb":144645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"24'3399","reported_seq":3445,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:04.108762+0000","last_change":"2024-08-19T13:15:04.699676+0000","last_active":"2024-08-19T13:19:04.108762+0000","last_peered":"2024-08-19T13:19:04.108762+0000","last_clean":"2024-08-19T13:19:04.108762+0000","last_became_active":"2024-08-19T13:15:04.699383+0000","last_became_peered":"2024-08-19T13:15:04.699383+0000","last_unstale":"2024-08-19T13:19:04.108762+0000","last_undegraded":"2024-08-19T13:19:04.108762+0000","last_fullsized":"2024-08-19T13:19:04.108762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3399,"log_dups_size":0,"ondisk_log_size":3399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:10:11.285767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":121,"num_read_kb":99,"num_write":3411,"num_write_kb":138057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"24'4377","reported_seq":4401,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.875182+0000","last_change":"2024-08-19T13:15:04.709245+0000","last_active":"2024-08-19T13:19:03.875182+0000","last_peered":"2024-08-19T13:19:03.875182+0000","last_clean":"2024-08-19T13:19:03.875182+0000","last_became_active":"2024-08-19T13:15:04.708174+0000","last_became_peered":"2024-08-19T13:15:04.708174+0000","last_unstale":"2024-08-19T13:19:03.875182+0000","last_undegraded":"2024-08-19T13:19:03.875182+0000","last_fullsized":"2024-08-19T13:19:03.875182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4377,"log_dups_size":0,"ondisk_log_size":4377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:25:57.887618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123076608,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":27,"num_read_kb":82,"num_write":4364,"num_write_kb":209707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"24'4912","reported_seq":5031,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:04.113539+0000","last_change":"2024-08-19T13:15:04.707739+0000","last_active":"2024-08-19T13:19:04.113539+0000","last_peered":"2024-08-19T13:19:04.113539+0000","last_clean":"2024-08-19T13:19:04.113539+0000","last_became_active":"2024-08-19T13:15:04.707482+0000","last_became_peered":"2024-08-19T13:15:04.707482+0000","last_unstale":"2024-08-19T13:19:04.113539+0000","last_undegraded":"2024-08-19T13:19:04.113539+0000","last_fullsized":"2024-08-19T13:19:04.113539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4912,"log_dups_size":0,"ondisk_log_size":4912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:07:38.674944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123340320,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":329,"num_read_kb":257,"num_write":5195,"num_write_kb":204415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"24'3582","reported_seq":3599,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.908548+0000","last_change":"2024-08-19T13:15:04.707186+0000","last_active":"2024-08-19T13:19:03.908548+0000","last_peered":"2024-08-19T13:19:03.908548+0000","last_clean":"2024-08-19T13:19:03.908548+0000","last_became_active":"2024-08-19T13:15:04.706891+0000","last_became_peered":"2024-08-19T13:15:04.706891+0000","last_unstale":"2024-08-19T13:19:03.908548+0000","last_undegraded":"2024-08-19T13:19:03.908548+0000","last_fullsized":"2024-08-19T13:19:03.908548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3582,"log_dups_size":0,"ondisk_log_size":3582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:45:10.115661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114560672,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3532,"num_write_kb":190257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"24'2895","reported_seq":2912,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.912954+0000","last_change":"2024-08-19T13:15:04.704399+0000","last_active":"2024-08-19T13:19:03.912954+0000","last_peered":"2024-08-19T13:19:03.912954+0000","last_clean":"2024-08-19T13:19:03.912954+0000","last_became_active":"2024-08-19T13:15:04.704113+0000","last_became_peered":"2024-08-19T13:15:04.704113+0000","last_unstale":"2024-08-19T13:19:03.912954+0000","last_undegraded":"2024-08-19T13:19:03.912954+0000","last_fullsized":"2024-08-19T13:19:03.912954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":2895,"log_dups_size":0,"ondisk_log_size":2895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:09:37.265840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103415808,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2845,"num_write_kb":157315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"22'4138","reported_seq":4160,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.874807+0000","last_change":"2024-08-19T13:15:04.709668+0000","last_active":"2024-08-19T13:19:03.874807+0000","last_peered":"2024-08-19T13:19:03.874807+0000","last_clean":"2024-08-19T13:19:03.874807+0000","last_became_active":"2024-08-19T13:15:04.709376+0000","last_became_peered":"2024-08-19T13:15:04.709376+0000","last_unstale":"2024-08-19T13:19:03.874807+0000","last_undegraded":"2024-08-19T13:19:03.874807+0000","last_fullsized":"2024-08-19T13:19:03.874807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:26:41.219376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":4141,"num_write_kb":181515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"24'3051","reported_seq":3084,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.914582+0000","last_change":"2024-08-19T13:15:04.704226+0000","last_active":"2024-08-19T13:19:03.914582+0000","last_peered":"2024-08-19T13:19:03.914582+0000","last_clean":"2024-08-19T13:19:03.914582+0000","last_became_active":"2024-08-19T13:15:04.703965+0000","last_became_peered":"2024-08-19T13:15:04.703965+0000","last_unstale":"2024-08-19T13:19:03.914582+0000","last_undegraded":"2024-08-19T13:19:03.914582+0000","last_fullsized":"2024-08-19T13:19:03.914582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":3051,"log_dups_size":0,"ondisk_log_size":3051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79700496,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":58,"num_read_kb":46,"num_write":3047,"num_write_kb":132115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"24'7856","reported_seq":8201,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:03.874875+0000","last_change":"2024-08-19T13:15:04.707304+0000","last_active":"2024-08-19T13:19:03.874875+0000","last_peered":"2024-08-19T13:19:03.874875+0000","last_clean":"2024-08-19T13:19:03.874875+0000","last_became_active":"2024-08-19T13:15:04.706952+0000","last_became_peered":"2024-08-19T13:15:04.706952+0000","last_unstale":"2024-08-19T13:19:03.874875+0000","last_undegraded":"2024-08-19T13:19:03.874875+0000","last_fullsized":"2024-08-19T13:19:03.874875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:15:03.684548+0000","last_clean_scrub_stamp":"2024-08-19T13:15:03.684548+0000","objects_scrubbed":0,"log_size":7856,"log_dups_size":0,"ondisk_log_size":7856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:27:03.627133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1110,"num_read_kb":992,"num_write":8069,"num_write_kb":173890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3575361784,"num_objects":879,"num_object_clones":0,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":879,"num_whiteouts":0,"num_read":2573,"num_read_kb":2589,"num_write":162480,"num_write_kb":5647601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6834126848,"data_stored":6849512568,"data_compressed":70382846,"data_compressed_allocated":73334784,"data_compressed_original":92987392,"omap_allocated":0,"internal_metadata":0},"log_size":165416,"ondisk_log_size":165416,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":58992,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1448376,"kb_used_data":1016044,"kb_used_omap":1,"kb_used_meta":432318,"kb_avail":92923464,"statfs":{"total":96636764160,"available":95153627136,"internally_reserved":0,"allocated":1040429056,"data_stored":1041941014,"data_compressed":9088911,"data_compressed_allocated":9474048,"data_compressed_original":11792384,"omap_allocated":1590,"internal_metadata":442694090},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443124,"kb_used_data":335592,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":93928716,"statfs":{"total":96636764160,"available":96183005184,"internally_reserved":0,"allocated":343646208,"data_stored":344910550,"data_compressed":5573327,"data_compressed_allocated":5816320,"data_compressed_original":7417856,"omap_allocated":1594,"internal_metadata":110098886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1250872,"kb_used_data":930988,"kb_used_omap":1,"kb_used_meta":319870,"kb_avail":93120968,"statfs":{"total":96636764160,"available":95355871232,"internally_reserved":0,"allocated":953331712,"data_stored":957667541,"data_compressed":10069671,"data_compressed_allocated":10522624,"data_compressed_original":15568896,"omap_allocated":1595,"internal_metadata":327547333},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1865984,"kb_used_data":1357368,"kb_used_omap":1,"kb_used_meta":508606,"kb_avail":92505856,"statfs":{"total":96636764160,"available":94725996544,"internally_reserved":0,"allocated":1389944832,"data_stored":1392671146,"data_compressed":17106045,"data_compressed_allocated":17801216,"data_compressed_original":21614592,"omap_allocated":1595,"internal_metadata":520812997},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1480520,"kb_used_data":1056508,"kb_used_omap":1,"kb_used_meta":423998,"kb_avail":92891320,"statfs":{"total":96636764160,"available":95120711680,"internally_reserved":0,"allocated":1081864192,"data_stored":1083353562,"data_compressed":11134625,"data_compressed_allocated":11599872,"data_compressed_original":13860864,"omap_allocated":1593,"internal_metadata":434174407},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496116,"kb_used_data":338664,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":93875724,"statfs":{"total":96636764160,"available":96128741376,"internally_reserved":0,"allocated":346791936,"data_stored":346713090,"data_compressed":1288748,"data_compressed_allocated":1335296,"data_compressed_original":1609728,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":874900,"kb_used_data":600844,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":93496940,"statfs":{"total":96636764160,"available":95740866560,"internally_reserved":0,"allocated":615264256,"data_stored":615354829,"data_compressed":3731208,"data_compressed_allocated":3866624,"data_compressed_original":4513792,"omap_allocated":1595,"internal_metadata":280623557},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1450176,"kb_used_data":1040056,"kb_used_omap":1,"kb_used_meta":410110,"kb_avail":92921664,"statfs":{"total":96636764160,"available":95151783936,"internally_reserved":0,"allocated":1065017344,"data_stored":1068710810,"data_compressed":12449303,"data_compressed_allocated":13000704,"data_compressed_original":17633280,"omap_allocated":1595,"internal_metadata":419953093},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1064701952,"data_stored":1068087872,"data_compressed":12419807,"data_compressed_allocated":12959744,"data_compressed_original":17121280,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":615006208,"data_stored":615260800,"data_compressed":3731208,"data_compressed_allocated":3866624,"data_compressed_original":4513792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346537984,"data_stored":346619080,"data_compressed":1288748,"data_compressed_allocated":1335296,"data_compressed_original":1609728,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1081610240,"data_stored":1083259552,"data_compressed":11134625,"data_compressed_allocated":11599872,"data_compressed_original":13860864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1389690880,"data_stored":1392577136,"data_compressed":17106045,"data_compressed_allocated":17801216,"data_compressed_original":21614592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":953073664,"data_stored":957573512,"data_compressed":10069671,"data_compressed_allocated":10522624,"data_compressed_original":15568896,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":343392256,"data_stored":344816540,"data_compressed":5573327,"data_compressed_allocated":5816320,"data_compressed_original":7417856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1040113664,"data_stored":1041318076,"data_compressed":9059415,"data_compressed_allocated":9433088,"data_compressed_original":11280384,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T13:19:10.778 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.779 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.779 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.779 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.779 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.779 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.779 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.779 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.779 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.779 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.780 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.780 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.780 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-08-19T13:14:55.033447+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=14, tm_sec=55, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.780 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.780 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-08-19T13:14:55.033447+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=14, tm_sec=55, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.780 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.781 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.781 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.781 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-08-19T13:14:55.033447+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=14, tm_sec=55, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.781 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.781 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.781 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-08-19T13:14:55.033447+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=14, tm_sec=55, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.781 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.781 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-08-19T13:14:55.033447+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=14, tm_sec=55, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.781 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-08-19T13:14:54.027963+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=14, tm_sec=54, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.782 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.782 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.782 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.782 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.782 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.782 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.782 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.782 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.783 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.783 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.783 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.783 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-08-19T13:15:03.684548+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=15, tm_sec=3, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=13, tm_min=19, tm_sec=5, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T13:19:10.783 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-08-19T13:19:30.784 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T13:19:30.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.894+0000 7f7a75944640 1 Processor -- start 2024-08-19T13:19:30.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.894+0000 7f7a75944640 1 -- start start 2024-08-19T13:19:30.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.895+0000 7f7a75944640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7a700b6270 0x7f7a700b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:30.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.895+0000 7f7a75944640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a70146360 0x7f7a700af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:30.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.895+0000 7f7a75944640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a700ae990 con 0x7f7a700b5e90 2024-08-19T13:19:30.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.895+0000 7f7a75944640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f7a700aec70 con 0x7f7a700b6270 2024-08-19T13:19:30.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.895+0000 7f7a75944640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7a700af100 con 0x7f7a70146360 2024-08-19T13:19:30.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.895+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a70146360 0x7f7a700af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:30.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.895+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a70146360 0x7f7a700af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41290/0 (socket says 172.21.15.103:41290) 2024-08-19T13:19:30.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.895+0000 7f7a6f7fe640 1 -- 172.21.15.103:0/1924807107 learned_addr learned my addr 172.21.15.103:0/1924807107 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:30.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.895+0000 7f7a667fc640 1 --2- 172.21.15.103:0/1924807107 >> v2:172.21.15.122:3300/0 conn(0x7f7a700b6270 0x7f7a700b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:30.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.895+0000 7f7a6f7fe640 1 -- 172.21.15.103:0/1924807107 >> v2:172.21.15.122:3300/0 conn(0x7f7a700b6270 msgr2=0x7f7a700b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:30.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.895+0000 7f7a6f7fe640 1 --2- 172.21.15.103:0/1924807107 >> v2:172.21.15.122:3300/0 conn(0x7f7a700b6270 0x7f7a700b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:30.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.895+0000 7f7a6f7fe640 1 -- 172.21.15.103:0/1924807107 >> v1:172.21.15.122:6789/0 conn(0x7f7a700b5e90 legacy=0x7f7a70139520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:30.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.895+0000 7f7a6f7fe640 1 -- 172.21.15.103:0/1924807107 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a70139c30 con 0x7f7a70146360 2024-08-19T13:19:30.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.896+0000 7f7a6f7fe640 1 --2- 172.21.15.103:0/1924807107 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a70146360 0x7f7a700af620 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009e50 tx=0x7f7a6802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=485b22838d029554 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:30.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.896+0000 7f7a6e7fc640 1 -- 172.21.15.103:0/1924807107 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68036070 con 0x7f7a70146360 2024-08-19T13:19:30.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.896+0000 7f7a6e7fc640 1 -- 172.21.15.103:0/1924807107 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a68031600 con 0x7f7a70146360 2024-08-19T13:19:30.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.896+0000 7f7a6e7fc640 1 -- 172.21.15.103:0/1924807107 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a680319d0 con 0x7f7a70146360 2024-08-19T13:19:30.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.897+0000 7f7a75944640 1 -- 172.21.15.103:0/1924807107 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a70146360 msgr2=0x7f7a700af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:30.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.897+0000 7f7a75944640 1 --2- 172.21.15.103:0/1924807107 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a70146360 0x7f7a700af620 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009e50 tx=0x7f7a6802a680 comp rx=0 tx=0).stop 2024-08-19T13:19:30.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.897+0000 7f7a75944640 1 -- 172.21.15.103:0/1924807107 shutdown_connections 2024-08-19T13:19:30.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.897+0000 7f7a75944640 1 --2- 172.21.15.103:0/1924807107 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a70146360 0x7f7a700af620 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:30.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.897+0000 7f7a75944640 1 --2- 172.21.15.103:0/1924807107 >> v2:172.21.15.122:3300/0 conn(0x7f7a700b6270 0x7f7a700b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:30.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.897+0000 7f7a75944640 1 -- 172.21.15.103:0/1924807107 >> 172.21.15.103:0/1924807107 conn(0x7f7a70067d20 msgr2=0x7f7a700b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:30.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.897+0000 7f7a75944640 1 -- 172.21.15.103:0/1924807107 shutdown_connections 2024-08-19T13:19:30.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.897+0000 7f7a75944640 1 -- 172.21.15.103:0/1924807107 wait complete. 2024-08-19T13:19:30.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.898+0000 7f7a75944640 1 Processor -- start 2024-08-19T13:19:30.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.898+0000 7f7a75944640 1 -- start start 2024-08-19T13:19:30.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a75944640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a700b5e90 0x7f7a70072410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:30.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a75944640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a700b6270 0x7f7a70072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:30.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a75944640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a70146360 0x7f7a701db7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:30.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a75944640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a700afce0 con 0x7f7a70146360 2024-08-19T13:19:30.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a75944640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7a700afb60 con 0x7f7a700b5e90 2024-08-19T13:19:30.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a75944640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7a700afe60 con 0x7f7a700b6270 2024-08-19T13:19:30.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a667fc640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a700b6270 0x7f7a70072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:30.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a70146360 0x7f7a701db7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:30.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a667fc640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a700b6270 0x7f7a70072ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41304/0 (socket says 172.21.15.103:41304) 2024-08-19T13:19:30.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a667fc640 1 -- 172.21.15.103:0/1534146996 learned_addr learned my addr 172.21.15.103:0/1534146996 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:30.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a6effd640 1 --2- 172.21.15.103:0/1534146996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a700b5e90 0x7f7a70072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:30.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a667fc640 1 -- 172.21.15.103:0/1534146996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a700b5e90 msgr2=0x7f7a70072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:30.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a667fc640 1 --2- 172.21.15.103:0/1534146996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a700b5e90 0x7f7a70072410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:30.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a667fc640 1 -- 172.21.15.103:0/1534146996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a70146360 msgr2=0x7f7a701db7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:30.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a667fc640 1 --2- 172.21.15.103:0/1534146996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a70146360 0x7f7a701db7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:30.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a667fc640 1 -- 172.21.15.103:0/1534146996 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a70072ff0 con 0x7f7a700b6270 2024-08-19T13:19:30.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a6f7fe640 1 --2- 172.21.15.103:0/1534146996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a70146360 0x7f7a701db7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:30.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.899+0000 7f7a667fc640 1 --2- 172.21.15.103:0/1534146996 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a700b6270 0x7f7a70072ab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800d830 tx=0x7f7a5800dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:30.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.900+0000 7f7a6cff9640 1 -- 172.21.15.103:0/1534146996 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5800b6c0 con 0x7f7a700b6270 2024-08-19T13:19:30.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.900+0000 7f7a6cff9640 1 -- 172.21.15.103:0/1534146996 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a58019070 con 0x7f7a700b6270 2024-08-19T13:19:30.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.900+0000 7f7a75944640 1 -- 172.21.15.103:0/1534146996 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7013eb20 con 0x7f7a700b6270 2024-08-19T13:19:30.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.900+0000 7f7a6cff9640 1 -- 172.21.15.103:0/1534146996 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5800ba80 con 0x7f7a700b6270 2024-08-19T13:19:30.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.900+0000 7f7a75944640 1 -- 172.21.15.103:0/1534146996 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a701cefb0 con 0x7f7a700b6270 2024-08-19T13:19:30.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.901+0000 7f7a75944640 1 -- 172.21.15.103:0/1534146996 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a700b1200 con 0x7f7a700b6270 2024-08-19T13:19:30.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.901+0000 7f7a6cff9640 1 -- 172.21.15.103:0/1534146996 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a5801e460 con 0x7f7a700b6270 2024-08-19T13:19:30.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.902+0000 7f7a6cff9640 1 --2- 172.21.15.103:0/1534146996 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7a5006be30 0x7f7a5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:30.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.902+0000 7f7a6effd640 1 --2- 172.21.15.103:0/1534146996 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7a5006be30 0x7f7a5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:30.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.902+0000 7f7a6cff9640 1 -- 172.21.15.103:0/1534146996 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7f7a5808b2c0 con 0x7f7a700b6270 2024-08-19T13:19:30.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.902+0000 7f7a6effd640 1 --2- 172.21.15.103:0/1534146996 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7a5006be30 0x7f7a5006e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7a60001780 tx=0x7f7a60009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:30.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:30.906+0000 7f7a6cff9640 1 -- 172.21.15.103:0/1534146996 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7f7a580596f0 con 0x7f7a700b6270 2024-08-19T13:19:31.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.073+0000 7f7a75944640 1 -- 172.21.15.103:0/1534146996 --> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a70075a00 con 0x7f7a5006be30 2024-08-19T13:19:31.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.076+0000 7f7a6cff9640 1 -- 172.21.15.103:0/1534146996 <== mgr.4110 v2:172.21.15.103:6832/3235664023 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+118210 (secure 0 0 0) 0x7f7a60001420 con 0x7f7a5006be30 2024-08-19T13:19:31.079 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T13:19:31.079 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-08-19T13:19:31.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.079+0000 7f7a75944640 1 -- 172.21.15.103:0/1534146996 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7a5006be30 msgr2=0x7f7a5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:31.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.079+0000 7f7a75944640 1 --2- 172.21.15.103:0/1534146996 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7a5006be30 0x7f7a5006e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7a60001780 tx=0x7f7a60009040 comp rx=0 tx=0).stop 2024-08-19T13:19:31.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.079+0000 7f7a75944640 1 -- 172.21.15.103:0/1534146996 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a700b6270 msgr2=0x7f7a70072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:31.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.079+0000 7f7a75944640 1 --2- 172.21.15.103:0/1534146996 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a700b6270 0x7f7a70072ab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800d830 tx=0x7f7a5800dcf0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.079+0000 7f7a75944640 1 -- 172.21.15.103:0/1534146996 shutdown_connections 2024-08-19T13:19:31.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.079+0000 7f7a75944640 1 --2- 172.21.15.103:0/1534146996 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7f7a5006be30 0x7f7a5006e2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.079+0000 7f7a75944640 1 --2- 172.21.15.103:0/1534146996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a70146360 0x7f7a701db7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.079+0000 7f7a75944640 1 --2- 172.21.15.103:0/1534146996 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7a700b6270 0x7f7a70072ab0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.079+0000 7f7a75944640 1 --2- 172.21.15.103:0/1534146996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a700b5e90 0x7f7a70072410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.079+0000 7f7a75944640 1 -- 172.21.15.103:0/1534146996 >> 172.21.15.103:0/1534146996 conn(0x7f7a70067d20 msgr2=0x7f7a700b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:31.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.079+0000 7f7a75944640 1 -- 172.21.15.103:0/1534146996 shutdown_connections 2024-08-19T13:19:31.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.079+0000 7f7a75944640 1 -- 172.21.15.103:0/1534146996 wait complete. 2024-08-19T13:19:31.091 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-08-19T13:19:30.053052+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3575952171,"num_objects":883,"num_object_clones":0,"num_object_copies":1766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":883,"num_whiteouts":0,"num_read":2689,"num_read_kb":2689,"num_write":162641,"num_write_kb":5650019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166338,"ondisk_log_size":166338,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9318368,"kb_used_data":6676064,"kb_used_omap":12,"kb_used_meta":2642227,"kb_avail":745656352,"statfs":{"total":773094113280,"available":763552104448,"internally_reserved":0,"allocated":6836289536,"data_stored":6851322542,"data_compressed":70441838,"data_compressed_allocated":73416704,"data_compressed_original":94011392,"omap_allocated":12752,"internal_metadata":2705641008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003000"},"pg_stats":[{"pgid":"3.1f","version":"24'3574","reported_seq":3612,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:10.250597+0000","last_change":"2024-08-19T13:19:10.250597+0000","last_active":"2024-08-19T13:19:10.250597+0000","last_peered":"2024-08-19T13:19:10.250597+0000","last_clean":"2024-08-19T13:19:10.250597+0000","last_became_active":"2024-08-19T13:15:04.703897+0000","last_became_peered":"2024-08-19T13:15:04.703897+0000","last_unstale":"2024-08-19T13:19:10.250597+0000","last_undegraded":"2024-08-19T13:19:10.250597+0000","last_fullsized":"2024-08-19T13:19:10.250597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3574","last_scrub_stamp":"2024-08-19T13:19:10.250458+0000","last_deep_scrub":"24'3574","last_deep_scrub_stamp":"2024-08-19T13:19:10.250458+0000","last_clean_scrub_stamp":"2024-08-19T13:19:10.250458+0000","objects_scrubbed":23,"log_size":3574,"log_dups_size":0,"ondisk_log_size":3574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:16:33.504881+0000","scrub_duration":277,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":3566,"num_write_kb":193154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"24'5780","reported_seq":5811,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:13.275546+0000","last_change":"2024-08-19T13:19:13.275600+0000","last_active":"2024-08-19T13:19:13.275546+0000","last_peered":"2024-08-19T13:19:13.275546+0000","last_clean":"2024-08-19T13:19:13.275546+0000","last_became_active":"2024-08-19T13:15:04.701653+0000","last_became_peered":"2024-08-19T13:15:04.701653+0000","last_unstale":"2024-08-19T13:19:13.275546+0000","last_undegraded":"2024-08-19T13:19:13.275546+0000","last_fullsized":"2024-08-19T13:19:13.275546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5780","last_scrub_stamp":"2024-08-19T13:19:13.275439+0000","last_deep_scrub":"24'5780","last_deep_scrub_stamp":"2024-08-19T13:19:13.275439+0000","last_clean_scrub_stamp":"2024-08-19T13:19:13.275439+0000","objects_scrubbed":31,"log_size":5780,"log_dups_size":0,"ondisk_log_size":5780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:29:31.932332+0000","scrub_duration":335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128583504,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5733,"num_write_kb":196004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"24'5903","reported_seq":5931,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:16.123708+0000","last_change":"2024-08-19T13:19:16.123739+0000","last_active":"2024-08-19T13:19:16.123708+0000","last_peered":"2024-08-19T13:19:16.123708+0000","last_clean":"2024-08-19T13:19:16.123708+0000","last_became_active":"2024-08-19T13:15:04.707756+0000","last_became_peered":"2024-08-19T13:15:04.707756+0000","last_unstale":"2024-08-19T13:19:16.123708+0000","last_undegraded":"2024-08-19T13:19:16.123708+0000","last_fullsized":"2024-08-19T13:19:16.123708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5903","last_scrub_stamp":"2024-08-19T13:19:16.123641+0000","last_deep_scrub":"24'5903","last_deep_scrub_stamp":"2024-08-19T13:19:16.123641+0000","last_clean_scrub_stamp":"2024-08-19T13:19:16.123641+0000","objects_scrubbed":28,"log_size":5903,"log_dups_size":0,"ondisk_log_size":5903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:55:57.777159+0000","scrub_duration":236,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113312264,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5859,"num_write_kb":177354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2016","reported_seq":2042,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:15.029884+0000","last_change":"2024-08-19T13:19:15.029949+0000","last_active":"2024-08-19T13:19:15.029884+0000","last_peered":"2024-08-19T13:19:15.029884+0000","last_clean":"2024-08-19T13:19:15.029884+0000","last_became_active":"2024-08-19T13:15:04.706726+0000","last_became_peered":"2024-08-19T13:15:04.706726+0000","last_unstale":"2024-08-19T13:19:15.029884+0000","last_undegraded":"2024-08-19T13:19:15.029884+0000","last_fullsized":"2024-08-19T13:19:15.029884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'2016","last_scrub_stamp":"2024-08-19T13:19:15.029788+0000","last_deep_scrub":"24'2016","last_deep_scrub_stamp":"2024-08-19T13:19:15.029788+0000","last_clean_scrub_stamp":"2024-08-19T13:19:15.029788+0000","objects_scrubbed":18,"log_size":2016,"log_dups_size":0,"ondisk_log_size":2016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:09:23.069016+0000","scrub_duration":157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1990,"num_write_kb":94721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"24'3809","reported_seq":3837,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:12.298716+0000","last_change":"2024-08-19T13:19:12.298741+0000","last_active":"2024-08-19T13:19:12.298716+0000","last_peered":"2024-08-19T13:19:12.298716+0000","last_clean":"2024-08-19T13:19:12.298716+0000","last_became_active":"2024-08-19T13:15:04.706832+0000","last_became_peered":"2024-08-19T13:15:04.706832+0000","last_unstale":"2024-08-19T13:19:12.298716+0000","last_undegraded":"2024-08-19T13:19:12.298716+0000","last_fullsized":"2024-08-19T13:19:12.298716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3809","last_scrub_stamp":"2024-08-19T13:19:12.298654+0000","last_deep_scrub":"24'3809","last_deep_scrub_stamp":"2024-08-19T13:19:12.298654+0000","last_clean_scrub_stamp":"2024-08-19T13:19:12.298654+0000","objects_scrubbed":26,"log_size":3809,"log_dups_size":0,"ondisk_log_size":3809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:03:45.499047+0000","scrub_duration":331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104858112,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3741,"num_write_kb":132744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"24'7080","reported_seq":7111,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:11.052287+0000","last_change":"2024-08-19T13:19:11.052342+0000","last_active":"2024-08-19T13:19:11.052287+0000","last_peered":"2024-08-19T13:19:11.052287+0000","last_clean":"2024-08-19T13:19:11.052287+0000","last_became_active":"2024-08-19T13:15:04.706371+0000","last_became_peered":"2024-08-19T13:15:04.706371+0000","last_unstale":"2024-08-19T13:19:11.052287+0000","last_undegraded":"2024-08-19T13:19:11.052287+0000","last_fullsized":"2024-08-19T13:19:11.052287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'7080","last_scrub_stamp":"2024-08-19T13:19:11.052188+0000","last_deep_scrub":"24'7080","last_deep_scrub_stamp":"2024-08-19T13:19:11.052188+0000","last_clean_scrub_stamp":"2024-08-19T13:19:11.052188+0000","objects_scrubbed":32,"log_size":7080,"log_dups_size":0,"ondisk_log_size":7080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:14:29.031516+0000","scrub_duration":311,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134217728,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":7029,"num_write_kb":256635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"24'4638","reported_seq":4667,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:12.227463+0000","last_change":"2024-08-19T13:19:12.227492+0000","last_active":"2024-08-19T13:19:12.227463+0000","last_peered":"2024-08-19T13:19:12.227463+0000","last_clean":"2024-08-19T13:19:12.227463+0000","last_became_active":"2024-08-19T13:15:04.700993+0000","last_became_peered":"2024-08-19T13:15:04.700993+0000","last_unstale":"2024-08-19T13:19:12.227463+0000","last_undegraded":"2024-08-19T13:19:12.227463+0000","last_fullsized":"2024-08-19T13:19:12.227463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4638","last_scrub_stamp":"2024-08-19T13:19:12.227400+0000","last_deep_scrub":"24'4638","last_deep_scrub_stamp":"2024-08-19T13:19:12.227400+0000","last_clean_scrub_stamp":"2024-08-19T13:19:12.227400+0000","objects_scrubbed":27,"log_size":4638,"log_dups_size":0,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:29:10.990216+0000","scrub_duration":338,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109052416,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":4571,"num_write_kb":180994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"24'6397","reported_seq":6424,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:11.322779+0000","last_change":"2024-08-19T13:19:11.322804+0000","last_active":"2024-08-19T13:19:11.322779+0000","last_peered":"2024-08-19T13:19:11.322779+0000","last_clean":"2024-08-19T13:19:11.322779+0000","last_became_active":"2024-08-19T13:15:04.699219+0000","last_became_peered":"2024-08-19T13:15:04.699219+0000","last_unstale":"2024-08-19T13:19:11.322779+0000","last_undegraded":"2024-08-19T13:19:11.322779+0000","last_fullsized":"2024-08-19T13:19:11.322779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'6397","last_scrub_stamp":"2024-08-19T13:19:11.322707+0000","last_deep_scrub":"24'6397","last_deep_scrub_stamp":"2024-08-19T13:19:11.322707+0000","last_clean_scrub_stamp":"2024-08-19T13:19:11.322707+0000","objects_scrubbed":25,"log_size":6397,"log_dups_size":0,"ondisk_log_size":6397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:50:57.235692+0000","scrub_duration":367,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6366,"num_write_kb":136778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"24'7691","reported_seq":7722,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:09.379832+0000","last_change":"2024-08-19T13:19:09.379897+0000","last_active":"2024-08-19T13:19:09.379832+0000","last_peered":"2024-08-19T13:19:09.379832+0000","last_clean":"2024-08-19T13:19:09.379832+0000","last_became_active":"2024-08-19T13:15:04.700904+0000","last_became_peered":"2024-08-19T13:15:04.700904+0000","last_unstale":"2024-08-19T13:19:09.379832+0000","last_undegraded":"2024-08-19T13:19:09.379832+0000","last_fullsized":"2024-08-19T13:19:09.379832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'7691","last_scrub_stamp":"2024-08-19T13:19:09.379738+0000","last_deep_scrub":"24'7691","last_deep_scrub_stamp":"2024-08-19T13:19:09.379738+0000","last_clean_scrub_stamp":"2024-08-19T13:19:09.379738+0000","objects_scrubbed":36,"log_size":7691,"log_dups_size":0,"ondisk_log_size":7691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T15:35:32.861486+0000","scrub_duration":376,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146932240,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":7658,"num_write_kb":230670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"24'3916","reported_seq":3943,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:14.127378+0000","last_change":"2024-08-19T13:19:14.127414+0000","last_active":"2024-08-19T13:19:14.127378+0000","last_peered":"2024-08-19T13:19:14.127378+0000","last_clean":"2024-08-19T13:19:14.127378+0000","last_became_active":"2024-08-19T13:15:04.704383+0000","last_became_peered":"2024-08-19T13:15:04.704383+0000","last_unstale":"2024-08-19T13:19:14.127378+0000","last_undegraded":"2024-08-19T13:19:14.127378+0000","last_fullsized":"2024-08-19T13:19:14.127378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3916","last_scrub_stamp":"2024-08-19T13:19:14.127312+0000","last_deep_scrub":"24'3916","last_deep_scrub_stamp":"2024-08-19T13:19:14.127312+0000","last_clean_scrub_stamp":"2024-08-19T13:19:14.127312+0000","objects_scrubbed":26,"log_size":3916,"log_dups_size":0,"ondisk_log_size":3916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:12:39.895206+0000","scrub_duration":250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106366632,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3864,"num_write_kb":162549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"24'8325","reported_seq":8375,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:13.873723+0000","last_change":"2024-08-19T13:19:13.873753+0000","last_active":"2024-08-19T13:19:13.873723+0000","last_peered":"2024-08-19T13:19:13.873723+0000","last_clean":"2024-08-19T13:19:13.873723+0000","last_became_active":"2024-08-19T13:15:04.703277+0000","last_became_peered":"2024-08-19T13:15:04.703277+0000","last_unstale":"2024-08-19T13:19:13.873723+0000","last_undegraded":"2024-08-19T13:19:13.873723+0000","last_fullsized":"2024-08-19T13:19:13.873723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'8325","last_scrub_stamp":"2024-08-19T13:19:13.873658+0000","last_deep_scrub":"24'8325","last_deep_scrub_stamp":"2024-08-19T13:19:13.873658+0000","last_clean_scrub_stamp":"2024-08-19T13:19:13.873658+0000","objects_scrubbed":31,"log_size":8325,"log_dups_size":0,"ondisk_log_size":8325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:56:51.673638+0000","scrub_duration":265,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125960192,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":85,"num_read_kb":132,"num_write":8326,"num_write_kb":182642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"24'6482","reported_seq":6609,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:11.942343+0000","last_change":"2024-08-19T13:19:11.942470+0000","last_active":"2024-08-19T13:19:11.942343+0000","last_peered":"2024-08-19T13:19:11.942343+0000","last_clean":"2024-08-19T13:19:11.942343+0000","last_became_active":"2024-08-19T13:15:04.706496+0000","last_became_peered":"2024-08-19T13:15:04.706496+0000","last_unstale":"2024-08-19T13:19:11.942343+0000","last_undegraded":"2024-08-19T13:19:11.942343+0000","last_fullsized":"2024-08-19T13:19:11.942343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'6482","last_scrub_stamp":"2024-08-19T13:19:11.942207+0000","last_deep_scrub":"24'6482","last_deep_scrub_stamp":"2024-08-19T13:19:11.942207+0000","last_clean_scrub_stamp":"2024-08-19T13:19:11.942207+0000","objects_scrubbed":23,"log_size":6482,"log_dups_size":0,"ondisk_log_size":6482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T15:37:34.228862+0000","scrub_duration":208,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90964496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":6419,"num_write_kb":175583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:09.974720+0000","last_change":"2024-08-19T13:19:09.974720+0000","last_active":"2024-08-19T13:19:09.974720+0000","last_peered":"2024-08-19T13:19:09.974720+0000","last_clean":"2024-08-19T13:19:09.974720+0000","last_became_active":"2024-08-19T13:14:56.044662+0000","last_became_peered":"2024-08-19T13:14:56.044662+0000","last_unstale":"2024-08-19T13:19:09.974720+0000","last_undegraded":"2024-08-19T13:19:09.974720+0000","last_fullsized":"2024-08-19T13:19:09.974720+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:19:09.974589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:19:09.974589+0000","last_clean_scrub_stamp":"2024-08-19T13:19:09.974589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:42:15.781619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039371300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'6818","reported_seq":6855,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:08.246904+0000","last_change":"2024-08-19T13:19:08.246904+0000","last_active":"2024-08-19T13:19:08.246904+0000","last_peered":"2024-08-19T13:19:08.246904+0000","last_clean":"2024-08-19T13:19:08.246904+0000","last_became_active":"2024-08-19T13:15:04.700540+0000","last_became_peered":"2024-08-19T13:15:04.700540+0000","last_unstale":"2024-08-19T13:19:08.246904+0000","last_undegraded":"2024-08-19T13:19:08.246904+0000","last_fullsized":"2024-08-19T13:19:08.246904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'6818","last_scrub_stamp":"2024-08-19T13:19:08.246803+0000","last_deep_scrub":"24'6818","last_deep_scrub_stamp":"2024-08-19T13:19:08.246803+0000","last_clean_scrub_stamp":"2024-08-19T13:19:08.246803+0000","objects_scrubbed":26,"log_size":6818,"log_dups_size":0,"ondisk_log_size":6818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:16:21.566219+0000","scrub_duration":277,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104988672,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":34,"num_read_kb":151,"num_write":6787,"num_write_kb":141259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:07.841169+0000","last_change":"2024-08-19T13:19:07.841169+0000","last_active":"2024-08-19T13:19:07.841169+0000","last_peered":"2024-08-19T13:19:07.841169+0000","last_clean":"2024-08-19T13:19:07.841169+0000","last_became_active":"2024-08-19T13:14:56.045731+0000","last_became_peered":"2024-08-19T13:14:56.045731+0000","last_unstale":"2024-08-19T13:19:07.841169+0000","last_undegraded":"2024-08-19T13:19:07.841169+0000","last_fullsized":"2024-08-19T13:19:07.841169+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:19:07.841083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:19:07.841083+0000","last_clean_scrub_stamp":"2024-08-19T13:19:07.841083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T15:45:07.439179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044000000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"24'5417","reported_seq":5453,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:14.237699+0000","last_change":"2024-08-19T13:19:14.237739+0000","last_active":"2024-08-19T13:19:14.237699+0000","last_peered":"2024-08-19T13:19:14.237699+0000","last_clean":"2024-08-19T13:19:14.237699+0000","last_became_active":"2024-08-19T13:15:04.703011+0000","last_became_peered":"2024-08-19T13:15:04.703011+0000","last_unstale":"2024-08-19T13:19:14.237699+0000","last_undegraded":"2024-08-19T13:19:14.237699+0000","last_fullsized":"2024-08-19T13:19:14.237699+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5417","last_scrub_stamp":"2024-08-19T13:19:14.237634+0000","last_deep_scrub":"24'5417","last_deep_scrub_stamp":"2024-08-19T13:19:14.237634+0000","last_clean_scrub_stamp":"2024-08-19T13:19:14.237634+0000","objects_scrubbed":36,"log_size":5417,"log_dups_size":0,"ondisk_log_size":5417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:02:14.467679+0000","scrub_duration":346,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146276352,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5343,"num_write_kb":213509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":42,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:12.572709+0000","last_change":"2024-08-19T13:19:12.572731+0000","last_active":"2024-08-19T13:19:12.572709+0000","last_peered":"2024-08-19T13:19:12.572709+0000","last_clean":"2024-08-19T13:19:12.572709+0000","last_became_active":"2024-08-19T13:14:56.111427+0000","last_became_peered":"2024-08-19T13:14:56.111427+0000","last_unstale":"2024-08-19T13:19:12.572709+0000","last_undegraded":"2024-08-19T13:19:12.572709+0000","last_fullsized":"2024-08-19T13:19:12.572709+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:19:12.572641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:19:12.572641+0000","last_clean_scrub_stamp":"2024-08-19T13:19:12.572641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:28:03.940169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050374000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"24'8315","reported_seq":8343,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:10.311916+0000","last_change":"2024-08-19T13:19:10.311916+0000","last_active":"2024-08-19T13:19:10.311916+0000","last_peered":"2024-08-19T13:19:10.311916+0000","last_clean":"2024-08-19T13:19:10.311916+0000","last_became_active":"2024-08-19T13:15:04.698944+0000","last_became_peered":"2024-08-19T13:15:04.698944+0000","last_unstale":"2024-08-19T13:19:10.311916+0000","last_undegraded":"2024-08-19T13:19:10.311916+0000","last_fullsized":"2024-08-19T13:19:10.311916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'8315","last_scrub_stamp":"2024-08-19T13:19:10.311788+0000","last_deep_scrub":"24'8315","last_deep_scrub_stamp":"2024-08-19T13:19:10.311788+0000","last_clean_scrub_stamp":"2024-08-19T13:19:10.311788+0000","objects_scrubbed":34,"log_size":8315,"log_dups_size":0,"ondisk_log_size":8315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:56:41.954333+0000","scrub_duration":438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8278,"num_write_kb":279553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:08.848399+0000","last_change":"2024-08-19T13:19:08.848399+0000","last_active":"2024-08-19T13:19:08.848399+0000","last_peered":"2024-08-19T13:19:08.848399+0000","last_clean":"2024-08-19T13:19:08.848399+0000","last_became_active":"2024-08-19T13:14:56.111065+0000","last_became_peered":"2024-08-19T13:14:56.111065+0000","last_unstale":"2024-08-19T13:19:08.848399+0000","last_undegraded":"2024-08-19T13:19:08.848399+0000","last_fullsized":"2024-08-19T13:19:08.848399+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:19:08.848257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:19:08.848257+0000","last_clean_scrub_stamp":"2024-08-19T13:19:08.848257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:47:53.679166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042019699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"24'4312","reported_seq":4354,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:12.235481+0000","last_change":"2024-08-19T13:19:12.235481+0000","last_active":"2024-08-19T13:19:12.235481+0000","last_peered":"2024-08-19T13:19:12.235481+0000","last_clean":"2024-08-19T13:19:12.235481+0000","last_became_active":"2024-08-19T13:15:04.697675+0000","last_became_peered":"2024-08-19T13:15:04.697675+0000","last_unstale":"2024-08-19T13:19:12.235481+0000","last_undegraded":"2024-08-19T13:19:12.235481+0000","last_fullsized":"2024-08-19T13:19:12.235481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4312","last_scrub_stamp":"2024-08-19T13:19:12.235376+0000","last_deep_scrub":"24'4312","last_deep_scrub_stamp":"2024-08-19T13:19:12.235376+0000","last_clean_scrub_stamp":"2024-08-19T13:19:12.235376+0000","objects_scrubbed":30,"log_size":4312,"log_dups_size":0,"ondisk_log_size":4312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:40:30.497331+0000","scrub_duration":366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114253824,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":58,"num_read_kb":45,"num_write":4306,"num_write_kb":159578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"24'5405","reported_seq":5448,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:09.978914+0000","last_change":"2024-08-19T13:19:09.978914+0000","last_active":"2024-08-19T13:19:09.978914+0000","last_peered":"2024-08-19T13:19:09.978914+0000","last_clean":"2024-08-19T13:19:09.978914+0000","last_became_active":"2024-08-19T13:15:04.703417+0000","last_became_peered":"2024-08-19T13:15:04.703417+0000","last_unstale":"2024-08-19T13:19:09.978914+0000","last_undegraded":"2024-08-19T13:19:09.978914+0000","last_fullsized":"2024-08-19T13:19:09.978914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5405","last_scrub_stamp":"2024-08-19T13:19:09.978808+0000","last_deep_scrub":"24'5405","last_deep_scrub_stamp":"2024-08-19T13:19:09.978808+0000","last_clean_scrub_stamp":"2024-08-19T13:19:09.978808+0000","objects_scrubbed":28,"log_size":5405,"log_dups_size":0,"ondisk_log_size":5405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:34:37.150424+0000","scrub_duration":256,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":5405,"num_write_kb":165854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"24'3","reported_seq":45,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:10.899525+0000","last_change":"2024-08-19T13:19:10.899550+0000","last_active":"2024-08-19T13:19:10.899525+0000","last_peered":"2024-08-19T13:19:10.899525+0000","last_clean":"2024-08-19T13:19:10.899525+0000","last_became_active":"2024-08-19T13:14:56.045753+0000","last_became_peered":"2024-08-19T13:14:56.045753+0000","last_unstale":"2024-08-19T13:19:10.899525+0000","last_undegraded":"2024-08-19T13:19:10.899525+0000","last_fullsized":"2024-08-19T13:19:10.899525+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3","last_scrub_stamp":"2024-08-19T13:19:10.899445+0000","last_deep_scrub":"24'3","last_deep_scrub_stamp":"2024-08-19T13:19:10.899445+0000","last_clean_scrub_stamp":"2024-08-19T13:19:10.899445+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T15:59:17.832495+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00037158599999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"24'2846","reported_seq":2878,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:11.163391+0000","last_change":"2024-08-19T13:19:11.163473+0000","last_active":"2024-08-19T13:19:11.163391+0000","last_peered":"2024-08-19T13:19:11.163391+0000","last_clean":"2024-08-19T13:19:11.163391+0000","last_became_active":"2024-08-19T13:15:04.699255+0000","last_became_peered":"2024-08-19T13:15:04.699255+0000","last_unstale":"2024-08-19T13:19:11.163391+0000","last_undegraded":"2024-08-19T13:19:11.163391+0000","last_fullsized":"2024-08-19T13:19:11.163391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'2846","last_scrub_stamp":"2024-08-19T13:19:11.163291+0000","last_deep_scrub":"24'2846","last_deep_scrub_stamp":"2024-08-19T13:19:11.163291+0000","last_clean_scrub_stamp":"2024-08-19T13:19:11.163291+0000","objects_scrubbed":21,"log_size":2846,"log_dups_size":0,"ondisk_log_size":2846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:30:42.938165+0000","scrub_duration":253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84017152,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":2784,"num_write_kb":135939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:09.024609+0000","last_change":"2024-08-19T13:19:09.024609+0000","last_active":"2024-08-19T13:19:09.024609+0000","last_peered":"2024-08-19T13:19:09.024609+0000","last_clean":"2024-08-19T13:19:09.024609+0000","last_became_active":"2024-08-19T13:14:56.046524+0000","last_became_peered":"2024-08-19T13:14:56.046524+0000","last_unstale":"2024-08-19T13:19:09.024609+0000","last_undegraded":"2024-08-19T13:19:09.024609+0000","last_fullsized":"2024-08-19T13:19:09.024609+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:19:09.024522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:19:09.024522+0000","last_clean_scrub_stamp":"2024-08-19T13:19:09.024522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:31:08.377525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043587900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"24'5967","reported_seq":5996,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:13.351712+0000","last_change":"2024-08-19T13:19:13.351712+0000","last_active":"2024-08-19T13:19:13.351712+0000","last_peered":"2024-08-19T13:19:13.351712+0000","last_clean":"2024-08-19T13:19:13.351712+0000","last_became_active":"2024-08-19T13:15:04.709768+0000","last_became_peered":"2024-08-19T13:15:04.709768+0000","last_unstale":"2024-08-19T13:19:13.351712+0000","last_undegraded":"2024-08-19T13:19:13.351712+0000","last_fullsized":"2024-08-19T13:19:13.351712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5967","last_scrub_stamp":"2024-08-19T13:19:13.351585+0000","last_deep_scrub":"24'5967","last_deep_scrub_stamp":"2024-08-19T13:19:13.351585+0000","last_clean_scrub_stamp":"2024-08-19T13:19:13.351585+0000","objects_scrubbed":36,"log_size":5967,"log_dups_size":0,"ondisk_log_size":5967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:41:35.733857+0000","scrub_duration":373,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146866696,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5915,"num_write_kb":254104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":42,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:11.536510+0000","last_change":"2024-08-19T13:19:11.536532+0000","last_active":"2024-08-19T13:19:11.536510+0000","last_peered":"2024-08-19T13:19:11.536510+0000","last_clean":"2024-08-19T13:19:11.536510+0000","last_became_active":"2024-08-19T13:14:56.045391+0000","last_became_peered":"2024-08-19T13:14:56.045391+0000","last_unstale":"2024-08-19T13:19:11.536510+0000","last_undegraded":"2024-08-19T13:19:11.536510+0000","last_fullsized":"2024-08-19T13:19:11.536510+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T13:19:11.536446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T13:19:11.536446+0000","last_clean_scrub_stamp":"2024-08-19T13:19:11.536446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:56:54.779396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037776800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"24'5404","reported_seq":5449,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:09.221023+0000","last_change":"2024-08-19T13:19:09.221056+0000","last_active":"2024-08-19T13:19:09.221023+0000","last_peered":"2024-08-19T13:19:09.221023+0000","last_clean":"2024-08-19T13:19:09.221023+0000","last_became_active":"2024-08-19T13:15:04.698015+0000","last_became_peered":"2024-08-19T13:15:04.698015+0000","last_unstale":"2024-08-19T13:19:09.221023+0000","last_undegraded":"2024-08-19T13:19:09.221023+0000","last_fullsized":"2024-08-19T13:19:09.221023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5404","last_scrub_stamp":"2024-08-19T13:19:09.220956+0000","last_deep_scrub":"24'5404","last_deep_scrub_stamp":"2024-08-19T13:19:09.220956+0000","last_clean_scrub_stamp":"2024-08-19T13:19:09.220956+0000","objects_scrubbed":29,"log_size":5404,"log_dups_size":0,"ondisk_log_size":5404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:28:29.989810+0000","scrub_duration":266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":68,"num_read_kb":114,"num_write":5405,"num_write_kb":156434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"24'2","reported_seq":44,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:09.853464+0000","last_change":"2024-08-19T13:19:09.853489+0000","last_active":"2024-08-19T13:19:09.853464+0000","last_peered":"2024-08-19T13:19:09.853464+0000","last_clean":"2024-08-19T13:19:09.853464+0000","last_became_active":"2024-08-19T13:14:56.045464+0000","last_became_peered":"2024-08-19T13:14:56.045464+0000","last_unstale":"2024-08-19T13:19:09.853464+0000","last_undegraded":"2024-08-19T13:19:09.853464+0000","last_fullsized":"2024-08-19T13:19:09.853464+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'2","last_scrub_stamp":"2024-08-19T13:19:09.853398+0000","last_deep_scrub":"24'2","last_deep_scrub_stamp":"2024-08-19T13:19:09.853398+0000","last_clean_scrub_stamp":"2024-08-19T13:19:09.853398+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:17:13.643949+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00039910299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"24'119","reported_seq":191,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:10.551632+0000","last_change":"2024-08-19T13:19:10.551684+0000","last_active":"2024-08-19T13:19:10.551632+0000","last_peered":"2024-08-19T13:19:10.551632+0000","last_clean":"2024-08-19T13:19:10.551632+0000","last_became_active":"2024-08-19T13:14:55.043626+0000","last_became_peered":"2024-08-19T13:14:55.043626+0000","last_unstale":"2024-08-19T13:19:10.551632+0000","last_undegraded":"2024-08-19T13:19:10.551632+0000","last_fullsized":"2024-08-19T13:19:10.551632+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'119","last_scrub_stamp":"2024-08-19T13:19:10.551541+0000","last_deep_scrub":"24'119","last_deep_scrub_stamp":"2024-08-19T13:19:10.551541+0000","last_clean_scrub_stamp":"2024-08-19T13:19:10.551541+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:56:05.837676+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"24'2790","reported_seq":2817,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:10.245631+0000","last_change":"2024-08-19T13:19:10.245631+0000","last_active":"2024-08-19T13:19:10.245631+0000","last_peered":"2024-08-19T13:19:10.245631+0000","last_clean":"2024-08-19T13:19:10.245631+0000","last_became_active":"2024-08-19T13:15:04.698903+0000","last_became_peered":"2024-08-19T13:15:04.698903+0000","last_unstale":"2024-08-19T13:19:10.245631+0000","last_undegraded":"2024-08-19T13:19:10.245631+0000","last_fullsized":"2024-08-19T13:19:10.245631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'2790","last_scrub_stamp":"2024-08-19T13:19:10.245429+0000","last_deep_scrub":"24'2790","last_deep_scrub_stamp":"2024-08-19T13:19:10.245429+0000","last_clean_scrub_stamp":"2024-08-19T13:19:10.245429+0000","objects_scrubbed":20,"log_size":2790,"log_dups_size":0,"ondisk_log_size":2790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:50:11.887794+0000","scrub_duration":248,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2699,"num_write_kb":115587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"24'9977","reported_seq":10128,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:13.067920+0000","last_change":"2024-08-19T13:19:13.067920+0000","last_active":"2024-08-19T13:19:13.067920+0000","last_peered":"2024-08-19T13:19:13.067920+0000","last_clean":"2024-08-19T13:19:13.067920+0000","last_became_active":"2024-08-19T13:15:04.706442+0000","last_became_peered":"2024-08-19T13:15:04.706442+0000","last_unstale":"2024-08-19T13:19:13.067920+0000","last_undegraded":"2024-08-19T13:19:13.067920+0000","last_fullsized":"2024-08-19T13:19:13.067920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'9977","last_scrub_stamp":"2024-08-19T13:19:13.067756+0000","last_deep_scrub":"24'9977","last_deep_scrub_stamp":"2024-08-19T13:19:13.067756+0000","last_clean_scrub_stamp":"2024-08-19T13:19:13.067756+0000","objects_scrubbed":35,"log_size":9977,"log_dups_size":0,"ondisk_log_size":9977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:36:29.344998+0000","scrub_duration":339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145424384,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":386,"num_read_kb":352,"num_write":6877,"num_write_kb":212562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"24'5366","reported_seq":5397,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:12.279772+0000","last_change":"2024-08-19T13:19:12.279772+0000","last_active":"2024-08-19T13:19:12.279772+0000","last_peered":"2024-08-19T13:19:12.279772+0000","last_clean":"2024-08-19T13:19:12.279772+0000","last_became_active":"2024-08-19T13:15:04.707724+0000","last_became_peered":"2024-08-19T13:15:04.707724+0000","last_unstale":"2024-08-19T13:19:12.279772+0000","last_undegraded":"2024-08-19T13:19:12.279772+0000","last_fullsized":"2024-08-19T13:19:12.279772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5366","last_scrub_stamp":"2024-08-19T13:19:12.279685+0000","last_deep_scrub":"24'5366","last_deep_scrub_stamp":"2024-08-19T13:19:12.279685+0000","last_clean_scrub_stamp":"2024-08-19T13:19:12.279685+0000","objects_scrubbed":29,"log_size":5366,"log_dups_size":0,"ondisk_log_size":5366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:05:30.636515+0000","scrub_duration":327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":11,"num_read_kb":71,"num_write":5419,"num_write_kb":161478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"24'3569","reported_seq":3595,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:12.277544+0000","last_change":"2024-08-19T13:19:12.277609+0000","last_active":"2024-08-19T13:19:12.277544+0000","last_peered":"2024-08-19T13:19:12.277544+0000","last_clean":"2024-08-19T13:19:12.277544+0000","last_became_active":"2024-08-19T13:15:04.699228+0000","last_became_peered":"2024-08-19T13:15:04.699228+0000","last_unstale":"2024-08-19T13:19:12.277544+0000","last_undegraded":"2024-08-19T13:19:12.277544+0000","last_fullsized":"2024-08-19T13:19:12.277544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3569","last_scrub_stamp":"2024-08-19T13:19:12.277438+0000","last_deep_scrub":"24'3569","last_deep_scrub_stamp":"2024-08-19T13:19:12.277438+0000","last_clean_scrub_stamp":"2024-08-19T13:19:12.277438+0000","objects_scrubbed":22,"log_size":3569,"log_dups_size":0,"ondisk_log_size":3569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:34:54.881532+0000","scrub_duration":322,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3536,"num_write_kb":144645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"24'3423","reported_seq":3478,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:13.144789+0000","last_change":"2024-08-19T13:19:13.144868+0000","last_active":"2024-08-19T13:19:13.144789+0000","last_peered":"2024-08-19T13:19:13.144789+0000","last_clean":"2024-08-19T13:19:13.144789+0000","last_became_active":"2024-08-19T13:15:04.699383+0000","last_became_peered":"2024-08-19T13:15:04.699383+0000","last_unstale":"2024-08-19T13:19:13.144789+0000","last_undegraded":"2024-08-19T13:19:13.144789+0000","last_fullsized":"2024-08-19T13:19:13.144789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3423","last_scrub_stamp":"2024-08-19T13:19:13.144685+0000","last_deep_scrub":"24'3423","last_deep_scrub_stamp":"2024-08-19T13:19:13.144685+0000","last_clean_scrub_stamp":"2024-08-19T13:19:13.144685+0000","objects_scrubbed":24,"log_size":3423,"log_dups_size":0,"ondisk_log_size":3423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:39:53.945990+0000","scrub_duration":246,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":121,"num_read_kb":99,"num_write":3411,"num_write_kb":138057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"24'4409","reported_seq":4443,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:17.907301+0000","last_change":"2024-08-19T13:19:17.907405+0000","last_active":"2024-08-19T13:19:17.907301+0000","last_peered":"2024-08-19T13:19:17.907301+0000","last_clean":"2024-08-19T13:19:17.907301+0000","last_became_active":"2024-08-19T13:15:04.708174+0000","last_became_peered":"2024-08-19T13:15:04.708174+0000","last_unstale":"2024-08-19T13:19:17.907301+0000","last_undegraded":"2024-08-19T13:19:17.907301+0000","last_fullsized":"2024-08-19T13:19:17.907301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4409","last_scrub_stamp":"2024-08-19T13:19:17.907209+0000","last_deep_scrub":"24'4409","last_deep_scrub_stamp":"2024-08-19T13:19:17.907209+0000","last_clean_scrub_stamp":"2024-08-19T13:19:17.907209+0000","objects_scrubbed":32,"log_size":4409,"log_dups_size":0,"ondisk_log_size":4409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:46:33.486610+0000","scrub_duration":269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123076608,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":27,"num_read_kb":82,"num_write":4364,"num_write_kb":209707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"24'4946","reported_seq":5075,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:16.909274+0000","last_change":"2024-08-19T13:19:16.909344+0000","last_active":"2024-08-19T13:19:16.909274+0000","last_peered":"2024-08-19T13:19:16.909274+0000","last_clean":"2024-08-19T13:19:16.909274+0000","last_became_active":"2024-08-19T13:15:04.707482+0000","last_became_peered":"2024-08-19T13:15:04.707482+0000","last_unstale":"2024-08-19T13:19:16.909274+0000","last_undegraded":"2024-08-19T13:19:16.909274+0000","last_fullsized":"2024-08-19T13:19:16.909274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4946","last_scrub_stamp":"2024-08-19T13:19:16.909199+0000","last_deep_scrub":"24'4946","last_deep_scrub_stamp":"2024-08-19T13:19:16.909199+0000","last_clean_scrub_stamp":"2024-08-19T13:19:16.909199+0000","objects_scrubbed":34,"log_size":4946,"log_dups_size":0,"ondisk_log_size":4946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:52:10.732992+0000","scrub_duration":262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123340320,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":329,"num_read_kb":257,"num_write":5195,"num_write_kb":204415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"24'3610","reported_seq":3637,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:14.802051+0000","last_change":"2024-08-19T13:19:14.802079+0000","last_active":"2024-08-19T13:19:14.802051+0000","last_peered":"2024-08-19T13:19:14.802051+0000","last_clean":"2024-08-19T13:19:14.802051+0000","last_became_active":"2024-08-19T13:15:04.706891+0000","last_became_peered":"2024-08-19T13:15:04.706891+0000","last_unstale":"2024-08-19T13:19:14.802051+0000","last_undegraded":"2024-08-19T13:19:14.802051+0000","last_fullsized":"2024-08-19T13:19:14.802051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3610","last_scrub_stamp":"2024-08-19T13:19:14.801984+0000","last_deep_scrub":"24'3610","last_deep_scrub_stamp":"2024-08-19T13:19:14.801984+0000","last_clean_scrub_stamp":"2024-08-19T13:19:14.801984+0000","objects_scrubbed":28,"log_size":3610,"log_dups_size":0,"ondisk_log_size":3610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:47:27.204534+0000","scrub_duration":229,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114560672,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3532,"num_write_kb":190257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"24'2920","reported_seq":2947,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:11.276172+0000","last_change":"2024-08-19T13:19:11.276195+0000","last_active":"2024-08-19T13:19:11.276172+0000","last_peered":"2024-08-19T13:19:11.276172+0000","last_clean":"2024-08-19T13:19:11.276172+0000","last_became_active":"2024-08-19T13:15:04.704113+0000","last_became_peered":"2024-08-19T13:15:04.704113+0000","last_unstale":"2024-08-19T13:19:11.276172+0000","last_undegraded":"2024-08-19T13:19:11.276172+0000","last_fullsized":"2024-08-19T13:19:11.276172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'2920","last_scrub_stamp":"2024-08-19T13:19:11.276108+0000","last_deep_scrub":"24'2920","last_deep_scrub_stamp":"2024-08-19T13:19:11.276108+0000","last_clean_scrub_stamp":"2024-08-19T13:19:11.276108+0000","objects_scrubbed":25,"log_size":2920,"log_dups_size":0,"ondisk_log_size":2920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T15:13:00.299581+0000","scrub_duration":295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103415808,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2845,"num_write_kb":157315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"24'4158","reported_seq":4189,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:18.792410+0000","last_change":"2024-08-19T13:19:18.792465+0000","last_active":"2024-08-19T13:19:18.792410+0000","last_peered":"2024-08-19T13:19:18.792410+0000","last_clean":"2024-08-19T13:19:18.792410+0000","last_became_active":"2024-08-19T13:15:04.709376+0000","last_became_peered":"2024-08-19T13:15:04.709376+0000","last_unstale":"2024-08-19T13:19:18.792410+0000","last_undegraded":"2024-08-19T13:19:18.792410+0000","last_fullsized":"2024-08-19T13:19:18.792410+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4158","last_scrub_stamp":"2024-08-19T13:19:18.792342+0000","last_deep_scrub":"24'4158","last_deep_scrub_stamp":"2024-08-19T13:19:18.792342+0000","last_clean_scrub_stamp":"2024-08-19T13:19:18.792342+0000","objects_scrubbed":20,"log_size":4158,"log_dups_size":0,"ondisk_log_size":4158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:48:57.723368+0000","scrub_duration":190,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":4141,"num_write_kb":181515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"24'3071","reported_seq":3112,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:11.253814+0000","last_change":"2024-08-19T13:19:11.253870+0000","last_active":"2024-08-19T13:19:11.253814+0000","last_peered":"2024-08-19T13:19:11.253814+0000","last_clean":"2024-08-19T13:19:11.253814+0000","last_became_active":"2024-08-19T13:15:04.703965+0000","last_became_peered":"2024-08-19T13:15:04.703965+0000","last_unstale":"2024-08-19T13:19:11.253814+0000","last_undegraded":"2024-08-19T13:19:11.253814+0000","last_fullsized":"2024-08-19T13:19:11.253814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3071","last_scrub_stamp":"2024-08-19T13:19:11.253741+0000","last_deep_scrub":"24'3071","last_deep_scrub_stamp":"2024-08-19T13:19:11.253741+0000","last_clean_scrub_stamp":"2024-08-19T13:19:11.253741+0000","objects_scrubbed":20,"log_size":3071,"log_dups_size":0,"ondisk_log_size":3071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:06:11.263458+0000","scrub_duration":242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79700496,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":58,"num_read_kb":46,"num_write":3047,"num_write_kb":132115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"24'7880","reported_seq":8234,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-19T13:19:15.850799+0000","last_change":"2024-08-19T13:19:15.850863+0000","last_active":"2024-08-19T13:19:15.850799+0000","last_peered":"2024-08-19T13:19:15.850799+0000","last_clean":"2024-08-19T13:19:15.850799+0000","last_became_active":"2024-08-19T13:15:04.706952+0000","last_became_peered":"2024-08-19T13:15:04.706952+0000","last_unstale":"2024-08-19T13:19:15.850799+0000","last_undegraded":"2024-08-19T13:19:15.850799+0000","last_fullsized":"2024-08-19T13:19:15.850799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'7880","last_scrub_stamp":"2024-08-19T13:19:15.850721+0000","last_deep_scrub":"24'7880","last_deep_scrub_stamp":"2024-08-19T13:19:15.850721+0000","last_clean_scrub_stamp":"2024-08-19T13:19:15.850721+0000","objects_scrubbed":24,"log_size":7880,"log_dups_size":0,"ondisk_log_size":7880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T00:40:59.875485+0000","scrub_duration":245,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1110,"num_read_kb":992,"num_write":8069,"num_write_kb":173890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3575361784,"num_objects":879,"num_object_clones":0,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":879,"num_whiteouts":0,"num_read":2573,"num_read_kb":2589,"num_write":162480,"num_write_kb":5647601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6834126848,"data_stored":6849512568,"data_compressed":70382846,"data_compressed_allocated":73334784,"data_compressed_original":92987392,"omap_allocated":0,"internal_metadata":0},"log_size":166214,"ondisk_log_size":166214,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":58992,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1448372,"kb_used_data":1016044,"kb_used_omap":1,"kb_used_meta":432318,"kb_avail":92923468,"statfs":{"total":96636764160,"available":95153631232,"internally_reserved":0,"allocated":1040429056,"data_stored":1041941014,"data_compressed":9088911,"data_compressed_allocated":9474048,"data_compressed_original":11792384,"omap_allocated":1590,"internal_metadata":442694090},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443120,"kb_used_data":335592,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":93928720,"statfs":{"total":96636764160,"available":96183009280,"internally_reserved":0,"allocated":343646208,"data_stored":344910550,"data_compressed":5573327,"data_compressed_allocated":5816320,"data_compressed_original":7417856,"omap_allocated":1594,"internal_metadata":110098886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1250868,"kb_used_data":930988,"kb_used_omap":1,"kb_used_meta":319870,"kb_avail":93120972,"statfs":{"total":96636764160,"available":95355875328,"internally_reserved":0,"allocated":953331712,"data_stored":957667541,"data_compressed":10069671,"data_compressed_allocated":10522624,"data_compressed_original":15568896,"omap_allocated":1595,"internal_metadata":327547333},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1865984,"kb_used_data":1357368,"kb_used_omap":1,"kb_used_meta":508606,"kb_avail":92505856,"statfs":{"total":96636764160,"available":94725996544,"internally_reserved":0,"allocated":1389944832,"data_stored":1392671146,"data_compressed":17106045,"data_compressed_allocated":17801216,"data_compressed_original":21614592,"omap_allocated":1595,"internal_metadata":520812997},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1480516,"kb_used_data":1056508,"kb_used_omap":1,"kb_used_meta":423998,"kb_avail":92891324,"statfs":{"total":96636764160,"available":95120715776,"internally_reserved":0,"allocated":1081864192,"data_stored":1083353562,"data_compressed":11134625,"data_compressed_allocated":11599872,"data_compressed_original":13860864,"omap_allocated":1593,"internal_metadata":434174407},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496112,"kb_used_data":338664,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":93875728,"statfs":{"total":96636764160,"available":96128745472,"internally_reserved":0,"allocated":346791936,"data_stored":346713090,"data_compressed":1288748,"data_compressed_allocated":1335296,"data_compressed_original":1609728,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":879060,"kb_used_data":600844,"kb_used_omap":1,"kb_used_meta":278206,"kb_avail":93492780,"statfs":{"total":96636764160,"available":95736606720,"internally_reserved":0,"allocated":615264256,"data_stored":615354829,"data_compressed":3731208,"data_compressed_allocated":3866624,"data_compressed_original":4513792,"omap_allocated":1595,"internal_metadata":284883397},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1454336,"kb_used_data":1040056,"kb_used_omap":1,"kb_used_meta":414270,"kb_avail":92917504,"statfs":{"total":96636764160,"available":95147524096,"internally_reserved":0,"allocated":1065017344,"data_stored":1068710810,"data_compressed":12449303,"data_compressed_allocated":13000704,"data_compressed_original":17633280,"omap_allocated":1595,"internal_metadata":424212933},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29496,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1064701952,"data_stored":1068087872,"data_compressed":12419807,"data_compressed_allocated":12959744,"data_compressed_original":17121280,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":615006208,"data_stored":615260800,"data_compressed":3731208,"data_compressed_allocated":3866624,"data_compressed_original":4513792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346537984,"data_stored":346619080,"data_compressed":1288748,"data_compressed_allocated":1335296,"data_compressed_original":1609728,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1081610240,"data_stored":1083259552,"data_compressed":11134625,"data_compressed_allocated":11599872,"data_compressed_original":13860864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1389690880,"data_stored":1392577136,"data_compressed":17106045,"data_compressed_allocated":17801216,"data_compressed_original":21614592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":953073664,"data_stored":957573512,"data_compressed":10069671,"data_compressed_allocated":10522624,"data_compressed_original":15568896,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":343392256,"data_stored":344816540,"data_compressed":5573327,"data_compressed_allocated":5816320,"data_compressed_original":7417856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1040113664,"data_stored":1041318076,"data_compressed":9059415,"data_compressed_allocated":9433088,"data_compressed_original":11280384,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T13:19:31.094 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-08-19T13:19:31.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.191+0000 7efccad3d640 1 Processor -- start 2024-08-19T13:19:31.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.191+0000 7efccad3d640 1 -- start start 2024-08-19T13:19:31.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.192+0000 7efccad3d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcc4067960 0x7efcc413d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:31.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.192+0000 7efccad3d640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7efcc413dcd0 0x7efcc41445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:31.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.192+0000 7efccad3d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc4067d40 con 0x7efcc40aa4e0 2024-08-19T13:19:31.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.192+0000 7efccad3d640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7efcc40aa1e0 con 0x7efcc413dcd0 2024-08-19T13:19:31.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.192+0000 7efccad3d640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efcc413e180 con 0x7efcc4067960 2024-08-19T13:19:31.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.192+0000 7efcc8ad9640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcc4067960 0x7efcc413d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:31.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.192+0000 7efcc92da640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7efcc413dcd0 0x7efcc41445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:31.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.192+0000 7efcc8ad9640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcc4067960 0x7efcc413d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41320/0 (socket says 172.21.15.103:41320) 2024-08-19T13:19:31.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.192+0000 7efcc8ad9640 1 -- 172.21.15.103:0/2863957824 learned_addr learned my addr 172.21.15.103:0/2863957824 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:31.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.192+0000 7efcc8ad9640 1 -- 172.21.15.103:0/2863957824 >> v2:172.21.15.103:3300/0 conn(0x7efcc413dcd0 msgr2=0x7efcc41445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T13:19:31.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.192+0000 7efcc8ad9640 1 --2- 172.21.15.103:0/2863957824 >> v2:172.21.15.103:3300/0 conn(0x7efcc413dcd0 0x7efcc41445f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.192+0000 7efcc8ad9640 1 -- 172.21.15.103:0/2863957824 >> v1:172.21.15.122:6789/0 conn(0x7efcc40aa4e0 legacy=0x7efcc4143ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:31.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.192+0000 7efcc8ad9640 1 -- 172.21.15.103:0/2863957824 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc40ab590 con 0x7efcc4067960 2024-08-19T13:19:31.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.192+0000 7efcc92da640 1 --2- 172.21.15.103:0/2863957824 >> v2:172.21.15.103:3300/0 conn(0x7efcc413dcd0 0x7efcc41445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T13:19:31.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.193+0000 7efcc8ad9640 1 --2- 172.21.15.103:0/2863957824 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcc4067960 0x7efcc413d790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009870 tx=0x7efcb402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b59a7664093e2c60 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:31.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.193+0000 7efcc37fe640 1 -- 172.21.15.103:0/2863957824 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4037070 con 0x7efcc4067960 2024-08-19T13:19:31.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.193+0000 7efcc37fe640 1 -- 172.21.15.103:0/2863957824 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb4032440 con 0x7efcc4067960 2024-08-19T13:19:31.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.193+0000 7efcc37fe640 1 -- 172.21.15.103:0/2863957824 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4032810 con 0x7efcc4067960 2024-08-19T13:19:31.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.194+0000 7efccad3d640 1 -- 172.21.15.103:0/2863957824 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcc4067960 msgr2=0x7efcc413d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:31.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.194+0000 7efccad3d640 1 --2- 172.21.15.103:0/2863957824 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcc4067960 0x7efcc413d790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009870 tx=0x7efcb402a1c0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.194+0000 7efccad3d640 1 -- 172.21.15.103:0/2863957824 shutdown_connections 2024-08-19T13:19:31.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.194+0000 7efccad3d640 1 --2- 172.21.15.103:0/2863957824 >> v2:172.21.15.103:3300/0 conn(0x7efcc413dcd0 0x7efcc41445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.194+0000 7efccad3d640 1 --2- 172.21.15.103:0/2863957824 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcc4067960 0x7efcc413d790 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.194+0000 7efccad3d640 1 -- 172.21.15.103:0/2863957824 >> 172.21.15.103:0/2863957824 conn(0x7efcc40b18e0 msgr2=0x7efcc40b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:31.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.194+0000 7efccad3d640 1 -- 172.21.15.103:0/2863957824 shutdown_connections 2024-08-19T13:19:31.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.194+0000 7efccad3d640 1 -- 172.21.15.103:0/2863957824 wait complete. 2024-08-19T13:19:31.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.195+0000 7efccad3d640 1 Processor -- start 2024-08-19T13:19:31.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.195+0000 7efccad3d640 1 -- start start 2024-08-19T13:19:31.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efccad3d640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcc4067960 0x7efcc4145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:31.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efccad3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc40aa4e0 0x7efcc4138050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:31.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efccad3d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc413dcd0 0x7efcc4138590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:31.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efccad3d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efcc4144cb0 con 0x7efcc40aa4e0 2024-08-19T13:19:31.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efccad3d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efcc4144b30 con 0x7efcc413dcd0 2024-08-19T13:19:31.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efccad3d640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efcc40a9d70 con 0x7efcc4067960 2024-08-19T13:19:31.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc8ad9640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcc4067960 0x7efcc4145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:31.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc40aa4e0 0x7efcc4138050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:31.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc8ad9640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcc4067960 0x7efcc4145880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:41322/0 (socket says 172.21.15.103:41322) 2024-08-19T13:19:31.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc8ad9640 1 -- 172.21.15.103:0/3914592822 learned_addr learned my addr 172.21.15.103:0/3914592822 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T13:19:31.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc92da640 1 --2- 172.21.15.103:0/3914592822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc413dcd0 0x7efcc4138590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:31.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc3fff640 1 -- 172.21.15.103:0/3914592822 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcc4067960 msgr2=0x7efcc4145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:31.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc3fff640 1 --2- 172.21.15.103:0/3914592822 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcc4067960 0x7efcc4145880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc3fff640 1 -- 172.21.15.103:0/3914592822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc413dcd0 msgr2=0x7efcc4138590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:31.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc3fff640 1 --2- 172.21.15.103:0/3914592822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc413dcd0 0x7efcc4138590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc3fff640 1 -- 172.21.15.103:0/3914592822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc4147d40 con 0x7efcc40aa4e0 2024-08-19T13:19:31.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc92da640 1 --2- 172.21.15.103:0/3914592822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc413dcd0 0x7efcc4138590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T13:19:31.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc3fff640 1 --2- 172.21.15.103:0/3914592822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc40aa4e0 0x7efcc4138050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efcb800c310 tx=0x7efcb800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:31.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc17fa640 1 -- 172.21.15.103:0/3914592822 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb80052d0 con 0x7efcc40aa4e0 2024-08-19T13:19:31.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc17fa640 1 -- 172.21.15.103:0/3914592822 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb8005490 con 0x7efcc40aa4e0 2024-08-19T13:19:31.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efccad3d640 1 -- 172.21.15.103:0/3914592822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc4145dc0 con 0x7efcc40aa4e0 2024-08-19T13:19:31.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efcc17fa640 1 -- 172.21.15.103:0/3914592822 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8007260 con 0x7efcc40aa4e0 2024-08-19T13:19:31.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.196+0000 7efccad3d640 1 -- 172.21.15.103:0/3914592822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc4146120 con 0x7efcc40aa4e0 2024-08-19T13:19:31.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.197+0000 7efccad3d640 1 -- 172.21.15.103:0/3914592822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc40b5ed0 con 0x7efcc40aa4e0 2024-08-19T13:19:31.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.198+0000 7efcc17fa640 1 -- 172.21.15.103:0/3914592822 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efcb8007400 con 0x7efcc40aa4e0 2024-08-19T13:19:31.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.198+0000 7efcc17fa640 1 --2- 172.21.15.103:0/3914592822 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7efcb006be60 0x7efcb006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T13:19:31.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.198+0000 7efcc17fa640 1 -- 172.21.15.103:0/3914592822 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5645+0+0 (secure 0 0 0) 0x7efcb808a190 con 0x7efcc40aa4e0 2024-08-19T13:19:31.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.198+0000 7efcc8ad9640 1 --2- 172.21.15.103:0/3914592822 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7efcb006be60 0x7efcb006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T13:19:31.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.199+0000 7efcc8ad9640 1 --2- 172.21.15.103:0/3914592822 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7efcb006be60 0x7efcb006e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efcb40090b0 tx=0x7efcb402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T13:19:31.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.202+0000 7efcc17fa640 1 -- 172.21.15.103:0/3914592822 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186747 (secure 0 0 0) 0x7efcc40b5ed0 con 0x7efcc40aa4e0 2024-08-19T13:19:31.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.367+0000 7efccad3d640 1 -- 172.21.15.103:0/3914592822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7efcc413d790 con 0x7efcc40aa4e0 2024-08-19T13:19:31.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.371+0000 7efcc17fa640 1 -- 172.21.15.103:0/3914592822 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7efcb8058530 con 0x7efcc40aa4e0 2024-08-19T13:19:31.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.371+0000 7efcc17fa640 1 -- 172.21.15.103:0/3914592822 <== mon.0 v2:172.21.15.103:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efcb805c830 con 0x7efcc40aa4e0 2024-08-19T13:19:31.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.373+0000 7efccad3d640 1 -- 172.21.15.103:0/3914592822 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7efcb006be60 msgr2=0x7efcb006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:31.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.373+0000 7efccad3d640 1 --2- 172.21.15.103:0/3914592822 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7efcb006be60 0x7efcb006e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efcb40090b0 tx=0x7efcb402e000 comp rx=0 tx=0).stop 2024-08-19T13:19:31.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.373+0000 7efccad3d640 1 -- 172.21.15.103:0/3914592822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc40aa4e0 msgr2=0x7efcc4138050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T13:19:31.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.373+0000 7efccad3d640 1 --2- 172.21.15.103:0/3914592822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc40aa4e0 0x7efcc4138050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efcb800c310 tx=0x7efcb800c7d0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.374+0000 7efccad3d640 1 -- 172.21.15.103:0/3914592822 shutdown_connections 2024-08-19T13:19:31.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.374+0000 7efccad3d640 1 --2- 172.21.15.103:0/3914592822 >> [v2:172.21.15.103:6832/3235664023,v1:172.21.15.103:6833/3235664023] conn(0x7efcb006be60 0x7efcb006e320 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.374+0000 7efccad3d640 1 --2- 172.21.15.103:0/3914592822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc413dcd0 0x7efcc4138590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.374+0000 7efccad3d640 1 --2- 172.21.15.103:0/3914592822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc40aa4e0 0x7efcc4138050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.374+0000 7efccad3d640 1 --2- 172.21.15.103:0/3914592822 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcc4067960 0x7efcc4145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T13:19:31.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.374+0000 7efccad3d640 1 -- 172.21.15.103:0/3914592822 >> 172.21.15.103:0/3914592822 conn(0x7efcc40b18e0 msgr2=0x7efcc414a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T13:19:31.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.374+0000 7efccad3d640 1 -- 172.21.15.103:0/3914592822 shutdown_connections 2024-08-19T13:19:31.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T13:19:31.374+0000 7efccad3d640 1 -- 172.21.15.103:0/3914592822 wait complete. 2024-08-19T13:19:31.387 INFO:teuthology.misc:Shutting down mds daemons... 2024-08-19T13:19:31.387 INFO:teuthology.misc:Shutting down osd daemons... 2024-08-19T13:19:31.387 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-08-19T13:19:31.387 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T13:19:31.445 INFO:tasks.ceph.osd.0:Stopped 2024-08-19T13:19:31.445 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-08-19T13:19:31.446 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T13:19:31.508 INFO:tasks.ceph.osd.1:Stopped 2024-08-19T13:19:31.509 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-08-19T13:19:31.509 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T13:19:31.546 INFO:tasks.ceph.osd.2:Stopped 2024-08-19T13:19:31.546 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-08-19T13:19:31.546 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T13:19:31.642 INFO:tasks.ceph.osd.3:Stopped 2024-08-19T13:19:31.642 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-08-19T13:19:31.642 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T13:19:31.698 INFO:tasks.ceph.osd.4:Stopped 2024-08-19T13:19:31.698 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-08-19T13:19:31.698 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T13:19:31.751 INFO:tasks.ceph.osd.5:Stopped 2024-08-19T13:19:31.751 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-08-19T13:19:31.751 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T13:19:31.790 INFO:tasks.ceph.osd.6:Stopped 2024-08-19T13:19:31.790 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-08-19T13:19:31.790 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T13:19:31.885 INFO:tasks.ceph.osd.7:Stopped 2024-08-19T13:19:31.885 INFO:teuthology.misc:Shutting down mgr daemons... 2024-08-19T13:19:31.885 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-08-19T13:19:31.885 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T13:19:31.911 INFO:tasks.ceph.mgr.y:Stopped 2024-08-19T13:19:31.911 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-08-19T13:19:31.911 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T13:19:31.937 INFO:tasks.ceph.mgr.x:Stopped 2024-08-19T13:19:31.937 INFO:teuthology.misc:Shutting down mon daemons... 2024-08-19T13:19:31.937 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-08-19T13:19:31.937 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T13:19:31.963 INFO:tasks.ceph.mon.a:Stopped 2024-08-19T13:19:31.963 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-08-19T13:19:31.963 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T13:19:32.014 INFO:tasks.ceph.mon.c:Stopped 2024-08-19T13:19:32.015 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-08-19T13:19:32.015 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T13:19:32.026 INFO:tasks.ceph.mon.b:Stopped 2024-08-19T13:19:32.026 INFO:tasks.ceph:Checking cluster log for badness... 2024-08-19T13:19:32.027 DEBUG:teuthology.orchestra.run.smithi103:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | head -n 1 2024-08-19T13:19:32.081 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T13:19:32.082 DEBUG:teuthology.orchestra.run.smithi103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-08-19T13:19:32.299 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T13:19:32.299 DEBUG:teuthology.orchestra.run.smithi103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-08-19T13:19:32.406 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T13:19:32.406 DEBUG:teuthology.orchestra.run.smithi103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-08-19T13:19:32.459 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T13:19:32.459 DEBUG:teuthology.orchestra.run.smithi103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-08-19T13:19:32.530 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi122.front.sepia.ceph.com 2024-08-19T13:19:32.530 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-08-19T13:19:32.743 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi122.front.sepia.ceph.com 2024-08-19T13:19:32.743 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-08-19T13:19:32.827 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi122.front.sepia.ceph.com 2024-08-19T13:19:32.827 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-08-19T13:19:32.889 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi122.front.sepia.ceph.com 2024-08-19T13:19:32.889 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-08-19T13:19:32.970 INFO:tasks.ceph:Archiving mon data... 2024-08-19T13:19:32.971 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi/7859586/data/mon.a.tgz 2024-08-19T13:19:32.972 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2024-08-19T13:19:32.987 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.QKZVk0cDXb 2024-08-19T13:19:32.988 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.QKZVk0cDXb 2024-08-19T13:19:33.114 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.QKZVk0cDXb 2024-08-19T13:19:33.198 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.QKZVk0cDXb is 281KB 2024-08-19T13:19:33.294 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.QKZVk0cDXb 2024-08-19T13:19:33.309 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi/7859586/data/mon.c.tgz 2024-08-19T13:19:33.309 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2024-08-19T13:19:33.363 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.sDuNknd1I3 2024-08-19T13:19:33.364 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.sDuNknd1I3 2024-08-19T13:19:33.475 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.sDuNknd1I3 2024-08-19T13:19:33.557 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.sDuNknd1I3 is 281KB 2024-08-19T13:19:33.652 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.sDuNknd1I3 2024-08-19T13:19:33.667 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi/7859586/data/mon.b.tgz 2024-08-19T13:19:33.667 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-08-19T13:19:33.686 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.0jKwTDRZTS 2024-08-19T13:19:33.686 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.0jKwTDRZTS 2024-08-19T13:19:33.817 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.0jKwTDRZTS 2024-08-19T13:19:33.903 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.0jKwTDRZTS is 281KB 2024-08-19T13:19:33.970 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.0jKwTDRZTS 2024-08-19T13:19:33.986 INFO:tasks.ceph:Cleaning ceph cluster... 2024-08-19T13:19:33.986 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-19T13:19:33.988 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-19T13:19:34.067 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-08-19T13:19:34.106 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-08-19T13:19:34.106 INFO:tasks.ceph:Archiving crash dumps... 2024-08-19T13:19:34.107 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi/7859586/remote/smithi103/crash 2024-08-19T13:19:34.108 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-19T13:19:34.133 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi/7859586/remote/smithi122/crash 2024-08-19T13:19:34.134 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-19T13:19:34.170 INFO:tasks.ceph:Compressing logs... 2024-08-19T13:19:34.170 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-19T13:19:34.177 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-19T13:19:34.199 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.25616.log 2024-08-19T13:19:34.201 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.25023.log 2024-08-19T13:19:34.201 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.libcephsqlite.25616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24479.log 2024-08-19T13:19:34.202 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.libcephsqlite.25023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24616.log 2024-08-19T13:19:34.202 INFO:teuthology.orchestra.run.smithi103.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.25616.log.gz 2024-08-19T13:19:34.202 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23943.log 2024-08-19T13:19:34.202 INFO:teuthology.orchestra.run.smithi103.stderr: 87.4%/var/log/ceph/ceph-client.admin.24616.log: -- replaced with /var/log/ceph/ceph-client.admin.24479.log.gz 2024-08-19T13:19:34.202 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23946.log 2024-08-19T13:19:34.202 INFO:teuthology.orchestra.run.smithi103.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.24616.log.gz 2024-08-19T13:19:34.202 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.23943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.25639.log 2024-08-19T13:19:34.202 INFO:teuthology.orchestra.run.smithi103.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.23943.log.gz 2024-08-19T13:19:34.202 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.23946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.25520.log 2024-08-19T13:19:34.202 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.libcephsqlite.25639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24940.log 2024-08-19T13:19:34.203 INFO:teuthology.orchestra.run.smithi103.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.23946.log.gz 2024-08-19T13:19:34.203 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.libcephsqlite.25520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24273.log 2024-08-19T13:19:34.223 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24940.log: 90.0% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.25520.log.gz 2024-08-19T13:19:34.223 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23765.log 2024-08-19T13:19:34.223 INFO:teuthology.orchestra.run.smithi103.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.24940.log.gz 2024-08-19T13:19:34.224 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24611.log 2024-08-19T13:19:34.224 INFO:teuthology.orchestra.run.smithi103.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.24273.log.gz 2024-08-19T13:19:34.224 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23546.log 2024-08-19T13:19:34.224 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.23765.log: /var/log/ceph/ceph-client.admin.24611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24618.log 2024-08-19T13:19:34.224 INFO:teuthology.orchestra.run.smithi103.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.23765.log.gz 2024-08-19T13:19:34.224 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.23546.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.24611.log.gz 2024-08-19T13:19:34.224 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26275.log 2024-08-19T13:19:34.224 INFO:teuthology.orchestra.run.smithi103.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.23546.log.gz 2024-08-19T13:19:34.224 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23572.log 2024-08-19T13:19:34.224 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24618.log: /var/log/ceph/ceph-client.admin.26275.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-08-19T13:19:34.224 INFO:teuthology.orchestra.run.smithi103.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.24618.log.gz 2024-08-19T13:19:34.224 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.23572.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.26275.log.gz 2024-08-19T13:19:34.224 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23723.log 2024-08-19T13:19:34.224 INFO:teuthology.orchestra.run.smithi103.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.23572.log.gz 2024-08-19T13:19:34.225 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24615.log 2024-08-19T13:19:34.225 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.23723.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-08-19T13:19:34.225 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24615.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.23723.log.gz 2024-08-19T13:19:34.225 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26527.log 2024-08-19T13:19:34.225 INFO:teuthology.orchestra.run.smithi103.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.24615.log.gz 2024-08-19T13:19:34.225 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23490.log 2024-08-19T13:19:34.225 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.26527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26485.log 2024-08-19T13:19:34.225 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.23490.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.26527.log.gz 2024-08-19T13:19:34.225 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24858.log 2024-08-19T13:19:34.225 INFO:teuthology.orchestra.run.smithi103.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.23490.log.gz 2024-08-19T13:19:34.225 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19968.log 2024-08-19T13:19:34.225 INFO:teuthology.orchestra.run.smithi103.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.26485.log.gz 2024-08-19T13:19:34.225 INFO:teuthology.orchestra.run.smithi103.stderr:gzip/var/log/ceph/ceph-client.admin.24858.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.26065.log 2024-08-19T13:19:34.225 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.19968.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.24858.log.gz 2024-08-19T13:19:34.226 INFO:teuthology.orchestra.run.smithi103.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.26653.log 2024-08-19T13:19:34.226 INFO:teuthology.orchestra.run.smithi103.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.19968.log.gz 2024-08-19T13:19:34.226 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26401.log 2024-08-19T13:19:34.226 INFO:teuthology.orchestra.run.smithi103.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.26065.log.gz 2024-08-19T13:19:34.226 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19655.log 2024-08-19T13:19:34.226 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26401.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.26653.log.gz 2024-08-19T13:19:34.226 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26611.log 2024-08-19T13:19:34.226 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.19655.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.26401.log.gz 2024-08-19T13:19:34.226 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26737.log 2024-08-19T13:19:34.226 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5/var/log/ceph/ceph-client.admin.26611.log: --verbose -- 92.5% -- replaced with /var/log/ceph/ceph-client.admin.19655.log.gz 2024-08-19T13:19:34.226 INFO:teuthology.orchestra.run.smithi103.stderr: /var/log/ceph/ceph-client.admin.25897.log 2024-08-19T13:19:34.226 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26737.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.26611.log.gz 2024-08-19T13:19:34.226 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24612.log 2024-08-19T13:19:34.227 INFO:teuthology.orchestra.run.smithi103.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.26737.log.gz 2024-08-19T13:19:34.227 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.25897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26569.log 2024-08-19T13:19:34.227 INFO:teuthology.orchestra.run.smithi103.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.25897.log.gz 2024-08-19T13:19:34.227 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24612.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-19T13:19:34.227 INFO:teuthology.orchestra.run.smithi103.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.24612.log.gz 2024-08-19T13:19:34.227 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26107.log 2024-08-19T13:19:34.227 INFO:teuthology.orchestra.run.smithi103.stderr:gzip 89.4%/var/log/ceph/ceph.audit.log: -- replaced with /var/log/ceph/ceph-client.admin.26569.log.gz 2024-08-19T13:19:34.227 INFO:teuthology.orchestra.run.smithi103.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.23947.log 2024-08-19T13:19:34.227 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26107.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 89.4% -- replaced with /var/log/ceph/ceph-client.admin.26107.log.gz 2024-08-19T13:19:34.227 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-08-19T13:19:34.227 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.23947.log: gzip -5 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-19T13:19:34.227 INFO:teuthology.orchestra.run.smithi103.stderr: --verbose -- /var/log/ceph/ceph-client.libcephsqlite.25707.log 2024-08-19T13:19:34.227 INFO:teuthology.orchestra.run.smithi103.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.23947.log.gz 2024-08-19T13:19:34.227 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23937.log 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.libcephsqlite.25707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26191.log 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.23937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26443.log 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.23937.log.gz 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.25747.log 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26443.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.26191.log.gz 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24281.log 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.26443.log.gz 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.libcephsqlite.25747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26777.log 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24281.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.24281.log.gz 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.25499.log 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26777.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.26777.log.gz 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr: 91.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23941.log 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.libcephsqlite.25747.log.gz 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.libcephsqlite.25499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24816.log 2024-08-19T13:19:34.228 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.23941.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.23941.log.gz 2024-08-19T13:19:34.229 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24609.log 2024-08-19T13:19:34.229 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24816.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.24816.log.gz 2024-08-19T13:19:34.229 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26359.log 2024-08-19T13:19:34.229 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24609.log: 91.8% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.25499.log.gz 2024-08-19T13:19:34.229 INFO:teuthology.orchestra.run.smithi103.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.24609.log.gz 2024-08-19T13:19:34.229 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-08-19T13:19:34.229 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26359.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-08-19T13:19:34.229 INFO:teuthology.orchestra.run.smithi103.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.26359.log.gz 2024-08-19T13:19:34.229 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24275.log 2024-08-19T13:19:34.234 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23938.log 2024-08-19T13:19:34.235 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24275.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.24275.log.gz 2024-08-19T13:19:34.237 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19734.log 2024-08-19T13:19:34.237 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19574.log 2024-08-19T13:19:34.241 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19774.log 2024-08-19T13:19:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.19734.log: /var/log/ceph/ceph-client.admin.19574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19534.log 2024-08-19T13:19:34.242 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.19734.log.gz 2024-08-19T13:19:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.19774.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.19574.log.gz 2024-08-19T13:19:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-08-19T13:19:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.19534.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.19774.log.gz 2024-08-19T13:19:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19614.log 2024-08-19T13:19:34.242 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7%/var/log/ceph/ceph-mgr.x.log: -- replaced with /var/log/ceph/ceph-client.admin.19534.log.gz 2024-08-19T13:19:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19654.log 2024-08-19T13:19:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.19614.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-19T13:19:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.19654.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.19614.log.gz 2024-08-19T13:19:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19694.log 2024-08-19T13:19:34.242 INFO:teuthology.orchestra.run.smithi122.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.19654.log.gz 2024-08-19T13:19:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-08-19T13:19:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.19694.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-08-19T13:19:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.4.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.19694.log.gz 2024-08-19T13:19:34.243 INFO:teuthology.orchestra.run.smithi122.stderr: 89.1%gzip -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-19T13:19:34.243 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-08-19T13:19:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-08-19T13:19:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-08-19T13:19:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-19T13:19:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19494.log 2024-08-19T13:19:34.244 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24983.log 2024-08-19T13:19:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.19494.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.19494.log.gz 2024-08-19T13:19:34.244 INFO:teuthology.orchestra.run.smithi122.stderr: 86.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-19T13:19:34.244 INFO:teuthology.orchestra.run.smithi122.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-08-19T13:19:34.255 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.23938.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.23938.log.gz 2024-08-19T13:19:34.255 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23681.log 2024-08-19T13:19:34.269 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24983.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26023.log 2024-08-19T13:19:34.269 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.23681.log: -- replaced with /var/log/ceph/ceph-client.admin.24983.log.gz 2024-08-19T13:19:34.270 INFO:teuthology.orchestra.run.smithi103.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.23681.log.gz 2024-08-19T13:19:34.287 INFO:teuthology.orchestra.run.smithi103.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-08-19T13:19:34.287 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.25684.log 2024-08-19T13:19:34.287 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26023.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.26023.log.gz 2024-08-19T13:19:34.287 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26233.log 2024-08-19T13:19:34.287 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.libcephsqlite.25684.log: 89.6% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.25684.log.gz 2024-08-19T13:19:34.287 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25981.log 2024-08-19T13:19:34.287 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26233.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.26233.log.gz 2024-08-19T13:19:34.287 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.13835.log 2024-08-19T13:19:34.287 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.25981.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.25981.log.gz 2024-08-19T13:19:34.287 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.25596.log 2024-08-19T13:19:34.288 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph.tmp-client.admin.13835.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.13835.log.gz 2024-08-19T13:19:34.288 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24280.log 2024-08-19T13:19:34.295 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.libcephsqlite.25596.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-08-19T13:19:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24280.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.24280.log.gz 94.8% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.25596.log.gz 2024-08-19T13:19:34.296 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-08-19T13:19:34.297 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.25564.log 2024-08-19T13:19:34.309 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24269.log 2024-08-19T13:19:34.313 INFO:teuthology.orchestra.run.smithi122.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-08-19T13:19:34.328 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.libcephsqlite.25564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25939.log 2024-08-19T13:19:34.328 INFO:teuthology.orchestra.run.smithi103.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.25564.log.gz 2024-08-19T13:19:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24269.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-19T13:19:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.25939.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.25939.log.gz 2024-08-19T13:19:34.341 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.24269.log.gz 2024-08-19T13:19:34.352 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.25541.log 2024-08-19T13:19:34.354 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph.log: 92.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-19T13:19:34.370 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20049.log 2024-08-19T13:19:34.371 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.libcephsqlite.25541.log: 89.8% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.25541.log.gz 2024-08-19T13:19:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.25665.log 2024-08-19T13:19:34.390 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.20049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23944.log 2024-08-19T13:19:34.392 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.libcephsqlite.25665.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.20049.log.gz 2024-08-19T13:19:34.398 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-08-19T13:19:34.408 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.23944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26695.log 2024-08-19T13:19:34.411 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-mon.c.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.23944.log.gz 2024-08-19T13:19:34.416 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24270.log 2024-08-19T13:19:34.425 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26149.log 2024-08-19T13:19:34.426 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24270.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.24270.log.gz 2024-08-19T13:19:34.429 INFO:teuthology.orchestra.run.smithi103.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.26695.log.gz 2024-08-19T13:19:34.443 INFO:teuthology.orchestra.run.smithi103.stderr: 96.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25855.log 2024-08-19T13:19:34.443 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26149.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.26149.log.gz 2024-08-19T13:19:34.453 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.libcephsqlite.25665.log.gz 2024-08-19T13:19:34.455 INFO:teuthology.orchestra.run.smithi103.stderr: 90.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24900.log 2024-08-19T13:19:34.455 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.25855.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.25855.log.gz 2024-08-19T13:19:34.455 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-08-19T13:19:34.455 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.25477.log 2024-08-19T13:19:34.456 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24900.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.24900.log.gz 2024-08-19T13:19:34.456 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24283.log 2024-08-19T13:19:34.458 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.libcephsqlite.25477.log: 90.0% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.25477.log.gz 2024-08-19T13:19:34.458 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26317.log 2024-08-19T13:19:34.458 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24283.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.24283.log.gz 2024-08-19T13:19:34.460 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24282.log 2024-08-19T13:19:34.460 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.26317.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.26317.log.gz 2024-08-19T13:19:34.460 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24619.log 2024-08-19T13:19:34.461 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24282.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.24282.log.gz 2024-08-19T13:19:34.461 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23945.log 2024-08-19T13:19:34.462 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.24619.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.24619.log.gz 2024-08-19T13:19:34.462 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.23945.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.23945.log.gz 2024-08-19T13:19:34.471 INFO:teuthology.orchestra.run.smithi103.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.25639.log.gz 2024-08-19T13:19:34.539 INFO:teuthology.orchestra.run.smithi103.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-08-19T13:19:34.617 INFO:teuthology.orchestra.run.smithi103.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.25707.log.gz 2024-08-19T13:19:36.335 INFO:teuthology.orchestra.run.smithi103.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.25023.log.gz 2024-08-19T13:19:39.031 INFO:teuthology.orchestra.run.smithi122.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-08-19T13:19:39.214 INFO:teuthology.orchestra.run.smithi103.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-08-19T13:19:43.793 INFO:teuthology.orchestra.run.smithi103.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-08-19T13:19:46.341 INFO:teuthology.orchestra.run.smithi122.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-08-19T13:19:50.292 INFO:teuthology.orchestra.run.smithi103.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-08-19T13:19:50.451 INFO:teuthology.orchestra.run.smithi103.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-08-19T13:19:50.453 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-08-19T13:19:50.453 INFO:teuthology.orchestra.run.smithi103.stderr:real 0m16.263s 2024-08-19T13:19:50.454 INFO:teuthology.orchestra.run.smithi103.stderr:user 0m48.386s 2024-08-19T13:19:50.454 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m1.606s 2024-08-19T13:19:51.682 INFO:teuthology.orchestra.run.smithi122.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-08-19T13:19:54.161 INFO:teuthology.orchestra.run.smithi122.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-08-19T13:19:54.162 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-08-19T13:19:54.163 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m19.935s 2024-08-19T13:19:54.163 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m52.653s 2024-08-19T13:19:54.163 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m1.613s 2024-08-19T13:19:54.163 INFO:tasks.ceph:Archiving logs... 2024-08-19T13:19:54.163 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi/7859586/remote/smithi103/log 2024-08-19T13:19:54.165 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-19T13:20:02.049 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi/7859586/remote/smithi122/log 2024-08-19T13:20:02.051 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-19T13:20:10.861 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-19T13:20:10.870 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-19T13:20:10.870 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-19T13:20:10.873 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-19T13:20:10.974 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-19T13:20:10.982 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-19T13:20:10.983 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T13:20:10.985 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T13:20:11.000 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-08-19T13:20:11.003 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-08-19T13:20:11.005 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T13:20:11.005 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-08-19T13:20:11.005 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 +295us[ +285us] +/- 52ms 2024-08-19T13:20:11.005 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -224us[ -224us] +/- 72ms 2024-08-19T13:20:11.005 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 -316us[ -326us] +/- 87ms 2024-08-19T13:20:11.005 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T13:20:11.009 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T13:20:11.009 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-08-19T13:20:11.009 INFO:teuthology.orchestra.run.smithi122.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 +305us[ +303us] +/- 52ms 2024-08-19T13:20:11.009 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -218us[ -220us] +/- 72ms 2024-08-19T13:20:11.009 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 -300us[ -302us] +/- 87ms 2024-08-19T13:20:11.009 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T13:20:11.009 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-19T13:20:11.018 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-19T13:20:11.018 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-19T13:20:11.027 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-19T13:20:11.100 DEBUG:teuthology.orchestra.run.smithi122:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-19T13:20:11.152 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T13:20:11.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T13:20:11.183 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T13:20:11.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T13:20:11.212 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-19T13:20:11.224 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-19T13:20:11.236 INFO:teuthology.task.internal:Duration was 559.725544 seconds 2024-08-19T13:20:11.236 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-19T13:20:11.271 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-19T13:20:11.271 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-19T13:20:11.274 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-19T13:20:11.317 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T13:20:11.318 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T13:20:11.709 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-19T13:20:11.709 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T13:20:11.709 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-19T13:20:11.774 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-08-19T13:20:11.774 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-19T13:20:11.805 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-19T13:20:11.805 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-19T13:20:11.817 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-19T13:20:11.874 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-19T13:20:11.874 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T13:20:11.887 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T13:20:11.975 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-19T13:20:11.986 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-19T13:20:11.987 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-19T13:20:12.013 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-19T13:20:12.038 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-19T13:20:12.048 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-19T13:20:12.056 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-19T13:20:12.077 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-08-19T13:20:12.102 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-08-19T13:20:12.155 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-19T13:20:12.170 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi103.front.sepia.ceph.com, flagging run as failed 2024-08-19T13:20:12.170 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-19T13:20:12.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T13:20:12.209 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-19T13:20:12.220 INFO:teuthology.task.internal:Transferring archived files... 2024-08-19T13:20:12.221 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi/7859586/remote/smithi103 2024-08-19T13:20:12.221 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-19T13:20:19.863 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2024-08-19T13:20:19.871 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/teuthology-2024-08-18_21:00:03-rados-squid-distro-default-smithi/7859586/remote/smithi103/coredump/1724073532.25707.core.gz: gzip compressed data, was "1724073532.25707.core", last modified: Mon Aug 19 13:20:12 2024, max compression, original size modulo 2^32 173174784 2024-08-19T13:20:19.871 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2024-08-19T13:20:20.686 INFO:teuthology.task.internal: core looks like: /tmp/tmpfp_b3qzm: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_', real uid: 1000, effective uid: 1000, real gid: 1333, effective gid: 1333, execfn: '/usr/bin/sqlite3', platform: 'x86_64' 2024-08-19T13:20:20.705 INFO:teuthology.task.internal: dump_program: sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_ 2024-08-19T13:20:20.705 DEBUG:teuthology.orchestra.run.smithi103:> which 'sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_' 2024-08-19T13:20:20.734 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: invalid option -- 'c' 2024-08-19T13:20:20.734 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: invalid option -- 'm' 2024-08-19T13:20:20.734 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: invalid option -- 'd' 2024-08-19T13:20:20.734 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: invalid option -- 'c' 2024-08-19T13:20:20.734 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: invalid option -- 'm' 2024-08-19T13:20:20.734 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: invalid option -- 'd' 2024-08-19T13:20:20.735 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: invalid option -- 'c' 2024-08-19T13:20:20.735 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: invalid option -- 'm' 2024-08-19T13:20:20.735 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: invalid option -- 'd' 2024-08-19T13:20:20.735 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: no .output in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-08-19T13:20:20.735 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: no null in (/dev) 2024-08-19T13:20:20.735 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: no .load in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-08-19T13:20:20.735 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: no libcephsqlite.so in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-08-19T13:20:20.735 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: no pragma in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-08-19T13:20:20.735 INFO:teuthology.orchestra.run.smithi103.stderr:/usr/bin/which: no journal_ in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-08-19T13:20:20.736 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-08-19T13:20:20.736 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/sqlite3 2024-08-19T13:20:20.736 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi103 with status 6: "which 'sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_'" 2024-08-19T13:20:20.738 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-19T13:20:20.748 INFO:teuthology.task.internal:Not uploading archives. 2024-08-19T13:20:20.748 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-19T13:20:20.771 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-19T13:20:20.771 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-19T13:20:20.779 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-19T13:20:20.795 INFO:teuthology.orchestra.run.smithi103.stdout: 262218 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Aug 19 13:20 /home/ubuntu/cephtest 2024-08-19T13:20:20.795 INFO:teuthology.orchestra.run.smithi103.stdout: 262219 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Aug 19 13:20 /home/ubuntu/cephtest/archive 2024-08-19T13:20:20.795 INFO:teuthology.orchestra.run.smithi103.stdout: 395287 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 13:20 /home/ubuntu/cephtest/archive/audit 2024-08-19T13:20:20.795 INFO:teuthology.orchestra.run.smithi103.stdout: 395570 52 -rw------- 1 ubuntu ubuntu 52742 Aug 19 13:20 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2024-08-19T13:20:20.795 INFO:teuthology.orchestra.run.smithi103.stdout: 262221 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 13:20 /home/ubuntu/cephtest/archive/syslog 2024-08-19T13:20:20.796 INFO:teuthology.orchestra.run.smithi103.stdout: 262258 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Aug 19 13:10 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-08-19T13:20:20.796 INFO:teuthology.orchestra.run.smithi103.stdout: 262222 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Aug 19 13:10 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-08-19T13:20:20.796 INFO:teuthology.orchestra.run.smithi103.stdout: 262223 56 -rw-r--r-- 1 ubuntu ubuntu 53964 Aug 19 13:20 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T13:20:20.796 INFO:teuthology.orchestra.run.smithi103.stdout: 262220 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 13:18 /home/ubuntu/cephtest/archive/coredump 2024-08-19T13:20:20.796 INFO:teuthology.orchestra.run.smithi103.stdout: 262323 32416 -rw------- 1 ubuntu ubuntu 173174784 Aug 19 13:18 /home/ubuntu/cephtest/archive/coredump/1724073532.25707.core 2024-08-19T13:20:20.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T13:20:20.797 INFO:teuthology.orchestra.run.smithi103.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-08-19T13:20:20.797 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi103 with status 6: "which 'sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_'" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi103 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-08-19T13:20:20.797 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-19T13:20:20.808 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-19T13:20:20.818 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi103 with status 6: "which 'sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_'" 2024-08-19T13:20:20.821 INFO:teuthology.run:Summary data: description: rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} tasks/libcephsqlite} duration: 559.7255437374115 failure_reason: Found coredumps on ubuntu@smithi103.front.sepia.ceph.com flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-08-19T13:20:20.821 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-19T13:20:20.832 INFO:teuthology.orchestra.run.smithi122.stdout: 262218 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Aug 19 13:20 /home/ubuntu/cephtest 2024-08-19T13:20:20.832 INFO:teuthology.orchestra.run.smithi122.stdout: 262219 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Aug 19 13:20 /home/ubuntu/cephtest/archive 2024-08-19T13:20:20.833 INFO:teuthology.orchestra.run.smithi122.stdout: 395287 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 13:20 /home/ubuntu/cephtest/archive/audit 2024-08-19T13:20:20.833 INFO:teuthology.orchestra.run.smithi122.stdout: 395570 48 -rw------- 1 ubuntu ubuntu 46703 Aug 19 13:20 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2024-08-19T13:20:20.833 INFO:teuthology.orchestra.run.smithi122.stdout: 262221 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 13:20 /home/ubuntu/cephtest/archive/syslog 2024-08-19T13:20:20.833 INFO:teuthology.orchestra.run.smithi122.stdout: 262258 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Aug 19 13:10 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-08-19T13:20:20.833 INFO:teuthology.orchestra.run.smithi122.stdout: 262222 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Aug 19 13:10 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-08-19T13:20:20.833 INFO:teuthology.orchestra.run.smithi122.stdout: 262223 52 -rw-r--r-- 1 ubuntu ubuntu 52766 Aug 19 13:20 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T13:20:20.834 INFO:teuthology.orchestra.run.smithi122.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-08-19T13:20:20.921 INFO:teuthology.run:FAIL